Feature Request: Support for Separate Leaderboards or Parallel Phases

[ToastyDom]

Hello everyone,

We are currently setting up a challenge and ran into a couple of scenarios where we think additional features could really help. I'd like to share our use case and suggest three possible new features or improvements or ask for help for workarounds!

Context

Our challenge has two phases (for now):

• Phase 1: A single task with standard submission and scoring.
• Phase 2: Two separate tasks that produce different results and require different evaluation procedures.

Feature Suggestions and Questions

1. Task Selection for Multi-Task Phases
   In Phase 2, where we have two tasks, we noticed that users submitting to the phase are automatically set to submit to both tasks by default. This makes it very likely for users to accidentally submit to both tasks at the same time, even if they only intended to participate in one.

   Could it be possible to have the task selection checkboxes unticked by default, so that the user has to make a conscious choice of which task(s) they want to submit to?

2. Separate Leaderboards for Multi-Task Phases
   Since Phase 2 has two tasks, it would be great to have separate leaderboards for each task, instead of the current setup where all task results are combined into a single leaderboard. Right now, the combined leaderboard has the same columns for both tasks, which doesn’t work well if the tasks produce completely different results.

   I saw that there’s an option to add a new leaderboard, but it’s currently grayed out and cannot be selected. After looking through other tickets, I found #1553 and #873, which mention that the current implementation for multiple leaderboards is too complicated. Is this something that might be addressed soon? If not, what would you suggest as a workaround for now?

3. Running Two Phases in Parallel
   If having separate leaderboards for tasks in the same phase isn’t possible, another option could be to run two phases in parallel within the same challenge. This way, Phase 2 could be split into two independent phases (one for each task), allowing for cleanly separated scoring. Currently, with parallel tasks, only the first task in line is scored after. However, it seems that currently, one phase has to end before the next can start.

   Would it be possible to enable running two phases in parallel within the same challenge?

---

If neither separate leaderboards nor parallel phases are possible or intended, what would you suggest as the best way to structure a challenge like ours? Specifically, we’d like to handle:

• A first phase with one simple task.
• A second phase with two separate tasks that are thematically connected but require different scoring procedures/leaderboards. Essentially, both tasks fall under the same challenge, with Task A being the primary challenge and Task B being an optional extra. Task B should have its own separate scoring, which is where the idea comes from of either having a separate leaderboard for each task or of having a parallel phase for each task.

Thank you for your help and the work that goes into Codabench! It's a great platform!

Best regards,
Domenic :)

[Didayolo]

Thank you for your ticket. Related issues:

• 1570

• 1553