search

motional / nuplan-devkit

The devkit of the nuPlan dataset.
https://www.nuplan.org
Other
703 stars 136 forks source link

To recreate the Challenge baseline given in Warm-Up phase (by Host_68305_Team) #153

Closed arunbalajeev closed 2 years ago

arunbalajeev commented 2 years ago

Thanks for providing the baseline. We are trying to replicate the baseline and participate in the NuPlan Challenge. We are quite excited about it and have some questions regarding that.

a. What is training set used for this baseline? Did you use ~51M scenarios(correct me if I am wrong) which is part of training set? Or could you please let us know how you sampled the training set (eg. 100000 scenarios from each scenario type)? b. Do you use raster model or vector model? c. What is the baseline numbers for the test set that is provided publicly, if you have it? This can be quite useful in making a comparison locally. d. Did you use all the augmentations that you have provided in the nuplan-devkit? e. Will you be providing the pretrained weights for the baseline.

These answers would be quite useful for us to train our models and participate in the challenge.

patk-motional commented 2 years ago

Hi @arunbalajeev,

a. We subsample split the data manually internally. You'd have to set this parameter scenario_filter.limit_total_scenarios to subsample the data. See our FAQ b. The submission for the warm-up stage is actually the SimplePlanner not an ML-based planner c. Can you clarify what you mean by "baseline numbers"? d. We provide the augmentations as examples of how you could implement one yourself. e. We don't have plans to release the weights yet.

Thank you for your interest and support of nuPlan :)

arunbalajeev commented 2 years ago

Thanks @patk-motional for the answers. It is quite useful for us. By "baseline numbers", I meant your warm-up stage model numbers on the test-set you have released.