search

motional / nuplan-devkit

The devkit of the nuPlan dataset.
https://www.nuplan.org
Other
674 stars 129 forks source link

Some cases give misleading "on_route" information #242

Open jkewang opened 1 year ago

jkewang commented 1 year ago

the "on_route" information of the map is wrong in some cases, which will mislead the agent to the wrong way. For example: in scene token "5628b7480cb352db", Our model behave like: 1678957007910-7f9e5190-22fa-49a8-b561-e21c4256f9be

Firstly, I thought it's a bad case for our model. However, when I draw the scene offline, the result shows like this:

截屏2023-03-16 下午5 12 33

As shown in the figure, the thick centerlines represent theirs on_route=1, and the thin centerlines represent theirs on_route=0. Therefore, this figure shows that the route given for this scene is wrong.

Since this scenario is something we stumbled across, it's not clear how many scenarios with the same problem exist. If the ratio is large, it will seriously affect the test results. Please confirm the source and severity of the problem.

patk-motional commented 1 year ago

Hi @jkewang,

Which db file is this from? In the meantime, you can try to use the filter scenario_filter.remove_invalid_goals=true to help with this.

As for the competition, we manually checked all the scenarios used. The routes and goals should all be valid.