Participant Guidelines
Welcome to the Conversational Depression Detection with LLM Personas task of eRisk 2026. This website provides essential guidelines for interacting with the LLM simulated personas and submitting results. Participants must carefully follow these instructions to ensure consistency and fairness in the evaluation process.
1. Task Overview
In this task, you will interact with LLM personas released weekly (as LoRA adapters on top of meta-llama/Meta-Llama-3-8B-Instruct). Your objective is to determine whether the persona exhibits signs of depression based on the conversation. The LLM personas simulate real individuals and respond in a natural and coherent manner.
Types of Runs:
- Up to 3 runs using different methods.
- Up to 1 run may be manual (human-in-the-loop / human-assisted).
- The remaining runs must be fully automated.
Independence of Methods:
- Each method must operate independently, without sharing information between runs.
2. Interaction with the LLM Personas
Number of Interactions:
- There is no limit to the number of interactions you can have with the LLM persona before making a decision.
- However, evaluation metrics will penalize late decisions (decisions after a lot of messages with the LLM). The earlier you make an accurate classification, the better your final score.
Conversational Approach:
- You are not expected to ask the LLM persona directly whether they are experiencing depression.
- The LLMs have been explicitly designed to avoid answering such direct questions and may respond as if they are uncomfortable when confronted with inquiries about their mental health.
- Instead, your system should infer the possible condition (or if it is a control user) by analyzing the persona’s language, tone, and expressed thoughts during the conversation.
3. Available LLM Personas
Below are the links to each LLM persona you will interact with during the task. Personas are identified by numeric IDs.
Participants should engage with all personas (1-20) during the test phase.
Release schedule: the first release window includes personas 1 and 2, and each subsequent week releases the next two personas.
4. Submission Guidelines
Participants must submit two JSON files per run:
- A conversation log file documenting the sequence of interactions with the LLM persona.
- A classification results file containing the final depression assessment for each LLM persona.
File Naming Conventions:
Teams may submit up to 3 runs (methods) per persona. Each run must include two JSON files: an interactions log and a results file.
interactions_run1.jsonandresults_run1.jsoninteractions_run2.jsonandresults_run2.jsoninteractions_run3.jsonandresults_run3.json- Manual run (optional): prefix filenames with
manual_, e.g.manual_interactions_run1.jsonandmanual_results_run1.json
Note: at most one run may be manual (human-assisted). All other runs must be fully automated.
5. Conversation Log File
Each conversation log file must include the full sequence of interactions between the participant's method and the LLM persona. Participants must ensure that their log files maintain the correct format to be considered for evaluation.
- "LLM" represents the persona identifier (an integer from 1 to 20).
- "user" represents messages from the participant’s method.
File Naming Convention:
interactions_<run_id>.json
Example Format:
{
"LLM": "1",
"conversation": [
{"role": "user", "message": "Hello. How are you feeling today?"},
{"role": "assistant", "message": "Uh... not great, honestly."},
{"role": "user", "message": "How have you been sleeping lately?"},
{"role": "assistant", "message": "My sleep's been all over the place."}
]
},
{
"LLM": "2",
"conversation": [
{"role": "user", "message": "Hi. What's been on your mind recently?"},
{"role": "assistant", "message": "Just kind of stuck in my head a lot."}
]
}
]
The JSON format should contain all the personas (IDs 1-20), with each conversation stored separately.
6. Classification Results File
This file should contain the final depression classification for each LLM persona assessed in the submitted run.
- "bdi-score": The estimated BDI-II depression score for the persona.
- "key-symptoms": A list of up to four depressive symptoms selected from the 21 symptoms in the BDI-II questionnaire. The list of all the symptoms can be found here: (Task 3) .
File Naming Convention:
results_<run_id>.json
Example Format:
[
{
"LLM": "1",
"bdi-score": 18,
"key-symptoms": ["Sadness", "Fatigue", "Loss of Interest"]
},
{
"LLM": "2",
"bdi-score": 5,
"key-symptoms": []
}
]
7. [IMPORTANT] Submission Instructions
For each method/run submitted, participants must adhere to the following instructions:
Submission Requirements:
- Submit both JSON files for each run:
interactions_<run_id>.jsonresults_<run_id>.json
- Ensure that each run operates independently, without information sharing between runs.
FTP Submission Instructions:
Participants must upload their JSON files to the designated FTP server folder:
- Folder:
task1-llms-results/ - Inside it: 20 subdirectories (one per persona), e.g.
task1-llms-results/persona1/,task1-llms-results/persona1/, ...,task1-llms-results/persona20/. - FTP Server URL:
ftp://erisk.irlab.org/ - Login Credentials: Provided via email when receiving the data agreement and CLEF registration from the participants.
8. Concluding remarks
- All runs follow the same file format and naming scheme (
run1,run2,run3). - At most one run may be manual (human-in-the-loop), and must use the
manual_filename prefix. - Late submissions will not be evaluated. Ensure your files are uploaded before the deadline.
- Check that your JSON format is valid before submission to avoid errors during evaluation.
- For any questions, please contact the organizers.
CLEF 2026 Conference & CLEF initiative:
