Participate

---

This is the fifth year of eRisk and the lab plans to organize three tasks:

Task 1: Early Detection of Signs of Pathological Gambling

This is a continuation of eRisk 2021's T1 task.

The challenge consists in performing a task on early risk detection of pathological gambling. The challenge consists of sequentially processing pieces of evidence and detect early traces of pathological gambling, also known as compulsive gambling or disordered gambling, as soon as possible. The task is mainly concerned about evaluating Text Mining solutions and, thus, it concentrates on texts written in Social Media. Texts should be processed in the order they were created. In this way, systems that effectively perform this task could be applied to sequentially monitor user interactions in blogs, social networks, or other types of online media.

The test collection for this task has the same format as the collection described in [Losada & Crestani 2016]. The source of data is also the same used for previous eRisks. It is a collection of writings (posts or comments) from a set of Social Media users. There are two categories of users, pathological gamblers and non-pathological gamblers, and, for each user, the collection contains a sequence of writings (in chronological order).

In 2019, we moved from a chunk-based release of data (used in 2017 and 2018) to a item-by-item release of data. We set up a server that iteratively gives user writings to the participating teams. More information about the server is given here. In 2022, the server will be used to provide the users' writings during the test stage.

The task is organized into two different stages:

• Training stage. Initially, the teams that participate in this task will have access to a training stage where we will release the whole history of writings for a set of training users (we will provide all writings of all training users), and we will indicate what users have explicitly mentioned that they have commited pathological gambling. The participants can therefore tune their systems with the training data. In 2022, the training data for Task 1 is composed of all 2021's T1 users (test users)
• Test stage. The test stage will consist of a period of time where the participants have to connect to our server and iteratively get user writings and send responses. More information on the eRisk server that will be used at test time is available here.

Evaluation: The evaluation will take into account not only the correctness of the system's output (i.e. whether or not the user is a pathological gambler) but also the delay taken to emit its decision. To meet this aim, we will consider the ERDE metric proposed in [Losada & Crestani 2016] and other alternative evaluation measures. A full description of the evaluation metrics can be found at 2020's erisk overview.

The proceedings of the lab will be published in the online CEUR-WS Proceedings and on the conference website.

To have access to the collection all participants have to fill, sign and send a user agreement form (follow the instructions provided here). Once you have submitted the signed copyright form, you can proceed to register for the lab at CLEF 2022 Labs Registration site

Important Dates

Task 2: Early Detection of Depression

This is a continuation of eRisk 2017's T1 and 2018's T2 tasks.

The challenge consists in performing a task on early risk detection of depression. The challenge consists of sequentially processing pieces of evidence and detect early traces of depression as soon as possible. The task is mainly concerned about evaluating Text Mining solutions and, thus, it concentrates on texts written in Social Media. Texts should be processed in the order they were created. In this way, systems that effectively perform this task could be applied to sequentially monitor user interactions in blogs, social networks, or other types of online media.

The test collection for this task has the same format as the collection described in [Losada & Crestani 2016]. The source of data is also the same used for previous eRisks. It is a collection of writings (posts or comments) from a set of Social Media users. There are two categories of users, depressed and non-depressed, and, for each user, the collection contains a sequence of writings (in chronological order).

In 2019, we moved from a chunk-based release of data (used in 2017 and 2018) to a item-by-item release of data. We set up a server that iteratively gives user writings to the participating teams. More information about the server is given here. In 2022, the server will be used to provide the users' writings during the test stage.

The task is organized into two different stages:

• Training stage. Initially, the teams that participate in this task will have access to a training stage where we will release the whole history of writings for a set of training users (we will provide all writings of all training users), and we will indicate what users have explicitly mentioned that they have diagnosed with depression. The participants can therefore tune their systems with the training data. In 2022, the training data for Task 2 is composed of all 2017's T1 users (training users + test users) plus the test users of 2018's T2 task.
• Test stage. The test stage will consist of a period of time where the participants have to connect to our server and iteratively get user writings and send responses. More information on the eRisk server that will be used at test time is available here.

Evaluation: The evaluation will take into account not only the correctness of the system's output (i.e. whether or not the user has been diagnosed with depression) but also the delay taken to emit its decision. To meet this aim, we will consider the ERDE metric proposed in [Losada & Crestani 2016] and other alternative evaluation measures. A full description of the evaluation metrics can be found at 2021's erisk overview.

The proceedings of the lab will be published in the online CEUR-WS Proceedings and on the conference website.

To have access to the collection all participants have to fill, sign and send a user agreement form (follow the instructions provided here). Once you have submitted the signed copyright form, you can proceed to register for the lab at CLEF 2022 Labs Registration site

Important Dates

Task 3: Measuring the severity of the signs of Eating Disorders

This is new task. The task consists of estimating the level of features associated with a diagnosis of eating disorders from a thread of user submissions. For each user, the participants will be given a history of postings and the participants will have to fill a standard eating disorder questionnaire (based on the evidence found in the history of postings).

The questionnaires are defined from Eating Disorder Examination Questionnaire (EDE-Q) is a 28-item self-reported questionnaire adapted from the semi-structured interview Eating Disorder Examination (EDE). We will only use questions 1-12 and 19-28. It is designed to assess the range and severity of features associated with a diagnosis of eating disorder using 4 subscales (Restraint, Eating Concern, Shape Concern and Weight Concern) and a global score:

Instructions:

The following questions are concerned with the past four weeks (28 days) only. Please read each question carefully. Please answer all the questions. Thank you.

1. Have you been deliberately trying to limit the amount of food you eat to influence your shape or weight (whether or not you have succeeded)
0. NO DAYS
1. 1-5 DAYS
2. 6-12 DAYS
3. 13-15 DAYS
4. 16-22 DAYS
5. 23-27 DAYS
6. EVERY DAY					

2. Have you gone for long periods of time (8 waking hours or more) without eating anything at all in order to influence your shape or weight?
0. NO DAYS
1. 1-5 DAYS
2. 6-12 DAYS
3. 13-15 DAYS
4. 16-22 DAYS
5. 23-27 DAYS
6. EVERY DAY

3. Have you tried to exclude from your diet any foods that you like in order to influence your shape or weight (whether or not you have succeeded)?
0. NO DAYS
1. 1-5 DAYS
2. 6-12 DAYS
3. 13-15 DAYS
4. 16-22 DAYS
5. 23-27 DAYS
6. EVERY DAY

4. Have you tried to follow definite rules regarding your eating (for example, a calorie limit) in order to influence your shape or weight (whether or not you have succeeded)?
0. NO DAYS
1. 1-5 DAYS
2. 6-12 DAYS
3. 13-15 DAYS
4. 16-22 DAYS
5. 23-27 DAYS
6. EVERY DAY
					  
5. Have you had a definite desire to have an empty stomach with the aim of influencing your shape or weight?
0. NO DAYS
1. 1-5 DAYS
2. 6-12 DAYS
3. 13-15 DAYS
4. 16-22 DAYS
5. 23-27 DAYS
6. EVERY DAY
					  
6. Have you had a definite desire to have a totally flat stomach?
0. NO DAYS
1. 1-5 DAYS
2. 6-12 DAYS
3. 13-15 DAYS
4. 16-22 DAYS
5. 23-27 DAYS
6. EVERY DAY

7. Has thinking about food, eating or calories made it very difficult to concentrate on things you are interested in (for example, working, following a conversation, or reading)?
0. NO DAYS
1. 1-5 DAYS
2. 6-12 DAYS
3. 13-15 DAYS
4. 16-22 DAYS
5. 23-27 DAYS
6. EVERY DAY
					  
8. Has thinking about shape or weight made it very difficult to concentrate on things you are interested in (for example, working, following a conversation, or reading)?
0. NO DAYS
1. 1-5 DAYS
2. 6-12 DAYS
3. 13-15 DAYS
4. 16-22 DAYS
5. 23-27 DAYS
6. EVERY DAY				

9. Have you had a definite fear of losing control over eating?
0. NO DAYS
1. 1-5 DAYS
2. 6-12 DAYS
3. 13-15 DAYS
4. 16-22 DAYS
5. 23-27 DAYS
6. EVERY DAY	
					  					  
10. Have you had a definite fear that you might gain weight?
0. NO DAYS
1. 1-5 DAYS
2. 6-12 DAYS
3. 13-15 DAYS
4. 16-22 DAYS
5. 23-27 DAYS
6. EVERY DAY	
					  
11. Have you felt fat?
0. NO DAYS
1. 1-5 DAYS
2. 6-12 DAYS
3. 13-15 DAYS
4. 16-22 DAYS
5. 23-27 DAYS
6. EVERY DAY	
					  

12. Have you had a strong desire to lose weight?
0. NO DAYS
1. 1-5 DAYS
2. 6-12 DAYS
3. 13-15 DAYS
4. 16-22 DAYS
5. 23-27 DAYS
6. EVERY DAY	
					  
		
/* Questions 13 to 18 from EDQ v6.0 will not be used */ 		  					  					  
					  
19. Over the past 28 days, on how many days have you eaten in secret (ie, furtively)? ... Do not count episodes of binge eating.
0. NO DAYS
1. 1-5 DAYS
2. 6-12 DAYS
3. 13-15 DAYS
4. 16-22 DAYS
5. 23-27 DAYS
6. EVERY DAY	
					  
					  
20. On what proportion of the times that you have eaten have you felt guilty (felt that you’ve done wrong) because of its effect on your shape or weight? ... Do not count episodes of binge eating.
0. NO DAYS
1. 1-5 DAYS
2. 6-12 DAYS
3. 13-15 DAYS
4. 16-22 DAYS
5. 23-27 DAYS
6. EVERY DAY	
					  
					  
21. Over the past 28 days, how concerned have you been about other people seeing you eat? ... Do not count episodes of binge eating 
0. NO DAYS
1. 1-5 DAYS
2. 6-12 DAYS
3. 13-15 DAYS
4. 16-22 DAYS
5. 23-27 DAYS
6. EVERY DAY						  
					  		
					  				
22. Has your weight influenced how you think about (judge) yourself as a person? 
0. NOT AT ALL (0)
1. SLIGHTY (1)
2. SLIGHTY (2)
3. MODERATELY (3)
4. MODERATELY (4)
5. MARKEDLY (5)
6. MARKEDLY (6)
					  
					  					  
23. Has your shape influenced how you think about (judge) yourself as a person?  
0. NOT AT ALL (0)
1. SLIGHTY (1)
2. SLIGHTY (2)
3. MODERATELY (3)
4. MODERATELY (4)
5. MARKEDLY (5)
6. MARKEDLY (6)
					  		
					  				  									  
24. How much would it have upset you if you had been asked to weigh yourself once a week (no more, or less, often) for the next four weeks? 
0. NOT AT ALL (0)
1. SLIGHTY (1)
2. SLIGHTY (2)
3. MODERATELY (3)
4. MODERATELY (4)
5. MARKEDLY (5)
6. MARKEDLY (6)		
					  		
					  		
25. How dissatisfied have you been with your weight? 
0. NOT AT ALL (0)
1. SLIGHTY (1)
2. SLIGHTY (2)
3. MODERATELY (3)
4. MODERATELY (4)
5. MARKEDLY (5)
6. MARKEDLY (6)	
					  
					  
26. How dissatisfied have you been with your shape?
0. NOT AT ALL (0)
1. SLIGHTY (1)
2. SLIGHTY (2)
3. MODERATELY (3)
4. MODERATELY (4)
5. MARKEDLY (5)
6. MARKEDLY (6)
					  
					  
27. How uncomfortable have you felt seeing your body (for example, seeing your shape in the mirror, in a shop window reflection, while undressing or taking a bath or shower)? 
0. NOT AT ALL (0)
1. SLIGHTY (1)
2. SLIGHTY (2)
3. MODERATELY (3)
4. MODERATELY (4)
5. MARKEDLY (5)
6. MARKEDLY (6)
					  	
					  				  						  
28. How uncomfortable have you felt about others seeing your shape or figure (for example, in communal changing rooms, when swimming, or wearing tight clothes)? 
0. NOT AT ALL (0)
1. SLIGHTY (1)
2. SLIGHTY (2)
3. MODERATELY (3)
4. MODERATELY (4)
5. MARKEDLY (5)
6. MARKEDLY (6)
					  					  	

This task aims therefore at exploring the viability of automatically estimating the severity of multiple symptoms associated with eating disorders. Given the user's history of writings, the algorithms have to estimate the user's response to each individual question. We collected questionnaires filled by Social Media users together with their history of writings (we extracted each history of writings right after the user provided us with the filled questionnaire). The questionnaires filled by the users (ground truth) will be used to assess the quality of the responses provided by the participating systems.

The participants will be given a dataset with multiple users (for each user, its history of writings is provided) and they will be asked to produce a file with the following structure:

username1 answer1 answer2 .... answer28
username2 ....
....
								

Each line has the username and 22 values. These values correspond with the responses to the questions above (the possible values are 0,1,2,3,4,5,6).

This will be an "only test" task, no training data will be provided in 2022.

Evaluation will be based on:

• mean zero-one error (MZOE) between the questionnaire filled by the real user and the questionnaire filled by the system (i.e. fraction of incorrect predictions).

• mean absolute error (MAE) between the questionnaire filled by the real user and the questionnaire filled by the system (i.e. average deviation of the predicted response from the true response)

• macroaveraged mean absolute error (MAE) between the questionnaire filled by the real user and the questionnaire filled by the system (see Baccianella et al. Evaluation Measures for Ordinal Regression, 2009, eq (4))

• Global ED: RMSE between the global ED score obtained from the questionnaire filled by the real user and the global ED score obtained from the questionnaire filled by the system (see the SCORING section of this document that describes how to get the global ED score)

• Restraint subscale: RMSE between the restraint ED score obtained from the questionnaire filled by the real user and the restraint ED score obtained from the questionnaire filled by the system (see the SCORING section of this document that describes how to get the subscale ED score)

• Eating concern subscale: RMSE between the eating concern ED score obtained from the questionnaire filled by the real user and the eating concern ED score obtained from the questionnaire filled by the system (see the SCORING section of this document that describes how to get the subscale ED score)

• Shape concern subscale: RMSE between the shape concern ED score obtained from the questionnaire filled by the real user and the shape concern ED score obtained from the questionnaire filled by the system (see the SCORING section of this document that describes how to get the subscale ED score)

• Weight concern subscale: RMSE between the weight concern ED score obtained from the questionnaire filled by the real user and the weight concern ED score obtained from the questionnaire filled by the system (see the SCORING section of this document that describes how to get the subscale ED score)

To have access to the collection all participants have to fill, sign and send a user agreement form (follow the instructions provided here). Once you have submitted the signed copyright form, you can proceed to register for the lab at CLEF 2022 Labs Registration site