search

codalab / codabench

Codabench is a flexible, easy-to-use and reproducible benchmarking platform. Check our paper at Patterns Cell Press https://hubs.li/Q01fwRWB0
Apache License 2.0
60 stars 26 forks source link

Incorrect daily submissions count #1469

Closed johanneskruse closed 3 weeks ago

johanneskruse commented 4 weeks ago

Hi,

I'm running a competition, and multiple participants have reached out to me with a problem: the number of submissions used for the day, "1 out of 2," will be displayed each day, even if they didn't make any submissions that day.

DC33D66E@75E06F29 151E6066

Is there an explanation or solution for this?

Best, Johannes

johanneskruse commented 4 weeks ago

My current theory is that even though it might have been submitted on day X, it was only evaluated the day after. Hence, the system would count it as submitted on that day. But I'm not sure.

Didayolo commented 3 weeks ago

Do you know if at some point the counter is reset? Maybe it is because the server time does not match their time zone.

johanneskruse commented 3 weeks ago

Participants keep having this issue, even if they haven't submitted for a long period:

image image
Didayolo commented 3 weeks ago

Users that experiences the problem: dknguyen176, chizhu, axi2017

johanneskruse commented 3 weeks ago

Users that experiences the problem: dknguyen176, chizhu, axi2017

These are just some of the users; many more have reached out with the same issue.

Didayolo commented 3 weeks ago

These are just some of the users; many more have reached out with the same issue.

Yes, I kept track of it to investigate the issue.

I found a fix and deployed it on production, it should be resolved now!

johanneskruse commented 3 weeks ago

@Didayolo - the displayed number seemed to be corrected but users can upload:

image
Didayolo commented 2 weeks ago

@johanneskruse Thank you for reporting this. I see the mistake in the code and will update it asap.

Didayolo commented 2 weeks ago

Here is the fix: https://github.com/codalab/codabench/commit/e3dbec5dac98dfc99543ed3c027a8acc048550a9

It is deployed and seems to work.