patk-motional
commented
1 year ago Hi @rohanb2018,
The simulator is there only for evaluation. We have not implemented closed-loop training. We do not have plans in the immediate future to add this to the devkit. We might if we have the bandwidth.
Data augmentation is used to account for closed-loop behavior. Hope that helps!
I'm a bit confused about the capabilities of the closed-loop simulator that comes with the nuPlan devkit.
From the description it seems that the simulator is mainly used during the evaluation phase - after an ML model has been learned, we can run the model in the closed-loop simulator to evaluate its performance (which I guess alternates between invoking the ML model based on the current state of the world, and updating the state of the world based on the ego-vehicle trajectory prediction that comes from the model).
What I'm wondering is - is it possible to use this simulator during the training phase as well? Or, based on other issues (like #130), is it the case that training is only open-loop? Meaning, that our submitted ML planners are constrained to only have access to the (static) dataset of (feature, trajectory) pairs, and must perform their training solely based on this dataset (without being able to generate new data with the help of the simulator)? Thanks so much!