search

codalab / codabench

Codabench is a flexible, easy-to-use and reproducible benchmarking platform. Check our paper at Patterns Cell Press https://hubs.li/Q01fwRWB0
Apache License 2.0
60 stars 26 forks source link

Documentation for submission limit reset #1434

Closed curious-broccoli closed 2 weeks ago

curious-broccoli commented 2 months ago

It is not clear to organizers and participants when the submission limit resets. Most people probably assume it resets at e.g. midnight in UTC or the server's time zone or at some other same set time every day. As stated in https://github.com/codalab/codabench/issues/1427#issuecomment-2084393604 the reset time seems to depend on each participant's previous submissions.

This information should be shown to at least the organizer (on site or maybe just in the wiki) and maybe to the user as well, so that they know when it will reset or that the reset time depends on when they submitted previously.

maybe https://github.com/codalab/codabench/blob/992bd86c661b157fde9ef1ed94ab35276694eddc/src/apps/competitions/models.py#L363

hieplpvip commented 2 months ago

Based on that line of code, shouldn't the submission limit reset at midnight?

ihsaan-ullah commented 2 months ago

Based on that line of code, shouldn't the submission limit reset at midnight?

Yes, but at the server midnight and not the participant's midnight