search

codalab / codabench

Codabench is a flexible, easy-to-use and reproducible benchmarking platform. Check our paper at Patterns Cell Press https://hubs.li/Q01fwRWB0
Apache License 2.0
76 stars 28 forks source link

Incorrect daily submissions count #1469

Closed johanneskruse closed 5 months ago

johanneskruse commented 5 months ago

Hi,

I'm running a competition, and multiple participants have reached out to me with a problem: the number of submissions used for the day, "1 out of 2," will be displayed each day, even if they didn't make any submissions that day.

DC33D66E@75E06F29 151E6066

Is there an explanation or solution for this?

Best, Johannes

johanneskruse commented 5 months ago

My current theory is that even though it might have been submitted on day X, it was only evaluated the day after. Hence, the system would count it as submitted on that day. But I'm not sure.

Didayolo commented 5 months ago

Do you know if at some point the counter is reset? Maybe it is because the server time does not match their time zone.

johanneskruse commented 5 months ago

Participants keep having this issue, even if they haven't submitted for a long period:

image image
Didayolo commented 5 months ago

Users that experiences the problem: dknguyen176, chizhu, axi2017

johanneskruse commented 5 months ago

Users that experiences the problem: dknguyen176, chizhu, axi2017

These are just some of the users; many more have reached out with the same issue.

Didayolo commented 5 months ago

These are just some of the users; many more have reached out with the same issue.

Yes, I kept track of it to investigate the issue.

I found a fix and deployed it on production, it should be resolved now!

johanneskruse commented 5 months ago

@Didayolo - the displayed number seemed to be corrected but users can upload:

image
Didayolo commented 5 months ago

@johanneskruse Thank you for reporting this. I see the mistake in the code and will update it asap.

Didayolo commented 5 months ago

Here is the fix: https://github.com/codalab/codabench/commit/e3dbec5dac98dfc99543ed3c027a8acc048550a9

It is deployed and seems to work.