Number of Submissions not updated

[kmin940]

Hello, we would like to inquire about the number of submissions not being updated.

Our challenge website is here. For some participants, the number of submissions used for the day is not reset even after the previous submission day has passed.

Could you please look into the problem?

Thank you for your support.

[ihsaan-ullah]

Please provide more details:

• Which participants are experiencing the issue
• When have they submitted their last submission

[kmin940]

Hi, Thank you for your prompt response. We have reached out to the participant inquiring about more details. We would like to ask one additional question. For a participant, they say it seems that most submissions keep getting stuck in the "Running" stage and never get to the "Finished" stage. 4 out of 6 had to be canceled.

We would like to inquire whether the number of submissions can be discounted if the job was canceled due to the job being suspended.

Thank you for your support.

[ihsaan-ullah]

The allowed number of submissions cannot be increased for a single participant. If you change it, it affects all the participants. By the way Cancelled/Failed submissions are not counted in the limit

[kmin940]

I see, thank you for your reply! Could the problem of jobs being getting stuck be resolved? Thank you.

[ihsaan-ullah]

Please share the IDs ofsubmission which are stuck. We will check them. Please also mention the queue name from your competition settings

[kmin940]

Thank you for your reply! The job IDs still in running status are as follows: 57960 57957 57947 57830 57795 57764 57761 57742 For the queue name, could you please let me know how to view the queue name? Currently in the Queue Management, there is no Queues.

[kmin940]

Hello,

Below are the details for the issue of the number of submissions not being reset.

Codabench Username of the participant: dknguyen176 On April 29th, the number of submissions was reset from 6 to 3 and the participant submitted three times to get to "6 out of 6". Today at 0:00 UTC, it's reset to "3 out of 6" again.

The job ID of the submissions are 57998, 57955, and 57860.

Thank you for your consideration!

[ihsaan-ullah]

I think there is a confusion in understanding this feature. I think you expect that at 00:00:00 submissions should reset but I think it resets individually for each participant i.e. for this participant it will reset on 2024-04-30 04:15.

In summary: if you submit a submission on January 01 at 10 AM, your submissions limit will reset on January 02 at 10 AM. If you think this is not happening, please let us know

[kmin940]

Thank you very much for your swift reply! I see, thank you for your answer! May I ask one last question? I think most of the jobs are in finished status, but for the following participant with job ID 57960, the status is still in running status.

I appreciate your continued support.

[ihsaan-ullah]

I think this submisison has some problem. @Didayolo can check problems with these stuck submissions in Running state.

• 57960
• 57957
• 57947
• 57830
• 57795
• 57764
• 57761
• 57742

[kmin940]

I see, thank you very much for your time and support!

[Didayolo]

A problem mentionned in #1205 is that when the compute worker crashes, the submission stays stuck in "Scoring" status without relevant logs.

I am trying to investigate the logs of the workers to find the errors, but it is a bit tricky as many submissions were computed in between.

[ff98li]

I think there is a confusion in understanding this feature. I think you expect that at 00:00:00 submissions should reset but I think it resets individually for each participant i.e. for this participant it will reset on 2024-04-30 04:15.

In summary: if you submit a submission on January 01 at 10 AM, your submissions limit will reset on January 02 at 10 AM. If you think this is not happening, please let us know

Hi Ihsan, thanks a ton for the detailed explanation! On top of this example, I have an additional question. Let's say if the participant had made separate submissions at 10:00 AM, 10:15 AM and 10:30 AM on January 1st (out of 6 daily quota), then would the daily submission quota reset to 2 at 10:00 AM on January 2nd, and then 0 at 10:30 AM, i.e. the daily limit is reset individually for each submission of that participant?

[hieplpvip]

Hi! I'm the user in the original post. The submission limit is still not reset for me.

The last submission I made was more than a day ago:

Yet the limit is now "4 out of 6":

My username is dknguyen176

[Didayolo]

I merge this issue with #1469 to keep track of it in one place.