search

motional / nuplan-devkit

The devkit of the nuPlan dataset.
https://www.nuplan.org
Other
673 stars 129 forks source link

Question about submission‘s evaluation execution time #247

Closed Shi-Qi-Li closed 1 year ago

Shi-Qi-Li commented 1 year ago

Hi team, How long does it take to get results after docker submitting?

patk-motional commented 1 year ago

Hi @Shi-Qi-Li,

We are manually triggering submission at the moment. Once triggered it should take around 2-3 hours for the results to be returned. In the future, this process will be automatic.

Shi-Qi-Li commented 1 year ago

Hi @Shi-Qi-Li,

We are manually triggering submission at the moment. Once triggered it should take around 2-3 hours for the results to be returned. In the future, this process will be automatic.

So the submission in Submitted status but does't return results because it has not been triggered?

patk-motional commented 1 year ago

Correct, this will become automatic soon

Shi-Qi-Li commented 1 year ago

Hi @Shi-Qi-Li,

We are manually triggering submission at the moment. Once triggered it should take around 2-3 hours for the results to be returned. In the future, this process will be automatic.

Hi @patk-motional , I have another question, I want to know if the result return is automatic? Because my submission has been running for a week now but no results have been returned yet.

patk-motional commented 1 year ago

Hi @Shi-Qi-Li,

Did your submission's status say submitted or running? If submitted then we have not triggered it yet.

Shi-Qi-Li commented 1 year ago

Hi @Shi-Qi-Li,

Did your submission's status say submitted or running? If submitted then we have not triggered it yet.

The status of my submission is running.

patk-motional commented 1 year ago

I see the submission now. Let us investigate

gianmarco-motional commented 1 year ago

@Shi-Qi-Li I believe you submitted the simulation container instead of your submission container. This caused a bad start on the execution and that's why it was marked running, but it actually failed almost immediately.

Shi-Qi-Li commented 1 year ago

@Shi-Qi-Li I believe you submitted the simulation container instead of your submission container. This caused a bad start on the execution and that's why it was marked running, but it actually failed almost immediately.

Thanks for your reply! I resubmitted the submission container, could you please trigger it?