Leaderboard form: a submission rule per challenge phase

codalab / codabench

Codabench is a flexible, easy-to-use and reproducible benchmarking platform. Check our paper at Patterns Cell Press https://hubs.li/Q01fwRWB0

Apache License 2.0

77 stars 28 forks source link

Leaderboard form: a submission rule per challenge phase #1331

Closed juliojj closed 9 months ago

juliojj commented 9 months ago

Hi Codabench developers, In "Leaderboard form" we can select, among other options, "Force last" or "Force best". It would be amazing if you could consider adding a functionality where we can select such option for each challenge phase, such that we could select, for example, to use "Force last" in a development phase, and "Force best" in a test (final) phase. Thanks and best

Didayolo commented 9 months ago

Dear Julio,

Thank you for the suggestion. I think we are not going to work on implementing this right now, because we are focusing on bug fixes. If several organizers ask for this feature, we will consider it.

In the meantime, I added it into the following discussion:

https://github.com/codalab/codabench/discussions/1334

Feel free to make other feature suggestions in the future. Best regards.