motional / nuplan-devkit

The devkit of the nuPlan dataset.
https://www.nuplan.org
Other
700 stars 136 forks source link

Questions about docker submission? #236

Closed Jeff09 closed 1 year ago

Jeff09 commented 1 year ago

Hi team,

I have two questions about the submission pipeline.

1.I have some error when running the submission tutorial? Here's the screenshot. image However, I have already set the env NUPLAN_MAPS_ROOT just like I set NUPLAN_DATA_ROOT and NUPLAN_EXP_ROOT. I don't know how to debug this, please help me out.

2.I also have a question about the entrypoint_submission.sh, how do we make sure the submission is valid? And is there any simulation output to check how the planner works?

Thank you so much.

gianmarco-motional commented 1 year ago

Hi @Jeff09! To be able to help you on the first question I need to understand if it's a problem related to the compose setup or not. Have you successfully run the tutorials?

Do you have the map data downloadd at that location? Or are your env vars pointing somewhere else?

About the second question, for the challenge we run our version of the simulation container, and your version of the submission container (launched by entrypoint_submission.sh), so the docker-compose setup should be representative of what we run on our server (as long as you haven't modified the simulation pipeline). You can inspect the performance through nuBoard (running the simulation with compose will generate the same artifacts as running via run_simulation.py.

gianmarco-motional commented 1 year ago

Hi @Jeff09! To be able to help you on the first question I need to understand if it's a problem related to the compose setup or not. Have you successfully run the tutorials?

Do you have the map data downloadd at that location? Or are your env vars pointing somewhere else?

About the second question, for the challenge we run our version of the simulation container, and your version of the submission container (launched by entrypoint_submission.sh), so the docker-compose setup should be representative of what we run on our server (as long as you haven't modified the simulation pipeline). You can inspect the performance through nuBoard (running the simulation with compose will generate the same artifacts as running via run_simulation.py.

Jeff09 commented 1 year ago

Hi @gianmarco-motional ,

Yes, I have successfuly run the tutorials. It looks like the compose setup is not right. When I run entrypoint_simulation.sh, it can load data location /data/sets/nuplan/, but can't set /data/sets/nuplan/maps.

For the second question, thank you for your answer.

Jeff09 commented 1 year ago

I think the issue is similar as 190.

gianmarco-motional commented 1 year ago

@Jeff09 I tested and for me it's working, however I have a particular setup which might affect this. Can you test if this change help? In the docker-compose.yml change

simulation:
    <<: *x-nuplan-volumes

to

simulation:
    <<: *x-nuplan-volumes
    <<: *x-nuplan-maps
Jeff09 commented 1 year ago

Hi @gianmarco-motional

I have changed the docker-compose.yaml, and the above error has gone. However, I had another error as the following screenshot shows. It can't find the db files in the docker even thought there are db files in the local folder. image For the local folder, it shows as follows. image

JingyuQian commented 1 year ago

@Jeff09 I tested and for me it's working, however I have a particular setup which might affect this. Can you test if this change help? In the docker-compose.yml change

simulation:
    <<: *x-nuplan-volumes

to

simulation:
    <<: *x-nuplan-volumes
    <<: *x-nuplan-maps

Hi @gianmarco-motional ,

The solution would overwrite mountings in x-nuplan-volumes with x-nuplan-maps, so only the maps root will be mounted on the container. I've created a #240 to address this issue. Please help verify, thanks!

gianmarco-motional commented 1 year ago

You are correct, my bad :slightly_smiling_face:

We will add this to the next release, in the meantime i'll check if I can merge your PR