Failure status: strange behaviour

[daviddanan]

Hi, i am one of the organizer of the following competition https://www.codabench.org/competitions/1534/

A competition participant reported a strange behaviour regarding the status. His submission went through the whole pipeline (ingestion program, scoring program), he even obtained a score but the status indicates "Failed". I don't understand why, it does not make sense to me, and there is no visible issue within the logs. Could you do something about it?

Thanks.

[Didayolo]

Hi, do you know the ID of the submission and the username of the submitter? Did it happen to other submissions?

[daviddanan]

Hi. Yes, it happenned for several submissions. Here is the complete list of submissions that failed like that so far:

username: 404error Ids that failed: 47879, 47703, 47702, 47594, 47592, 47591, 45824,

username: binhnk Ids: 45161,44932

[Didayolo]

For record: I am re-running submission 47879 (now with ID 48322).

[Didayolo]

Update: the re-run did not fail. I have currently no explanation on the "Failed" status on the first run.

We will try to investigate this problem. In the meantime, you can indicate your participants to submit again if an error occur.

[daviddanan]

This is quite strange because when i have a look at the submission it did fail as showed below. Even worse, there is no score, now.

According to the submitter, it fails afterward which seems to be consistent with your test. Any idea?

[Didayolo]

Oh sorry, you are right. I made a confusion between "48322" and "48332"

As you are using your own queue, I can't connect to the worker to find any interesting logs here.

I suggest that you:

• Re-launch one of the problematic submission
• Find the compute worker running the submissions in https://www.codabench.org/server_status
• Connect to the worker
• Inspect the logs associated to the submission (docker logs -f compute_worker)

[Didayolo]

I close this issue because it is very specific. Feel free to re-open it if needed.