search

codalab / codabench

Codabench is a flexible, easy-to-use and reproducible benchmarking platform. Check our paper at Patterns Cell Press https://hubs.li/Q01fwRWB0
Apache License 2.0
60 stars 26 forks source link

Competition dump missing settings #1481

Open curious-broccoli opened 3 weeks ago

curious-broccoli commented 3 weeks ago

(Running our own instance, which is a few commits behind.)

Competition dump does not include leaderboard's columns' precision and submission rule.

No columns' precision

Dumping a competition doesn't seem to include the leaderboard's columns' precision.

https://github.com/codalab/codabench/blob/6ff1c4532679c59b1fdb20168ee728b9a7f42ae0/src/apps/competitions/tasks.py#L93-L101

No submission rule

Dumping a competition doesn't seem to include the leaderboard's submission_rule. This seems to be intended (for now), see https://github.com/codalab/codabench/blob/6ff1c4532679c59b1fdb20168ee728b9a7f42ae0/src/apps/competitions/tasks.py#L82-L91