Time-consuming for closed-loop Evaluation

MCZhi / GameFormer-Planner

[ICCV & CVPR Workshop] Learning-enabled Interactive Prediction and Planning Framework for Autonomous Vehicles

https://mczhi.github.io/GameFormer/

MIT License

143 stars 17 forks source link

Time-consuming for closed-loop Evaluation #19

Closed JuggerNau closed 2 months ago

JuggerNau commented 4 months ago

Hi, I am trying your great work on nuplan dataset, but it seems really time-consuming to do the closed-loop evaluation (it takes about 1 minute for each senario, that means about 2 hours for each .db file , which is impossible to test on all validation set). If I made some mistakes?

MCZhi commented 4 months ago

Yes, it would normally take that amount of time (about 1 minute) to finish running a 15-second scenario. I would not suggest testing on all cases in the validation set but some representative examples.