search

Thinklab-SJTU / Bench2Drive

[NeurIPS 2024 Datasets and Benchmarks Track] Closed-Loop E2E-AD Benchmark Enhanced by World Model RL Expert
Other
1.3k stars 85 forks source link

Evaluation without sensor data recording #104

Closed dikshant2210 closed 1 month ago

dikshant2210 commented 2 months ago

Hello, is it possible to stop the sensor data logging when running the eval pipeline. I am trying to reproduce the results in the paper. So far, I have been able to run the eval on a single GPU which is quite slow and I would like to expedite it therefore I wanted to stop recording sensor data to disk but couldn't figure out how to do it.

jayyoung0802 commented 2 months ago

Hi, @dikshant2210, you can comment this line or change the save freq. Other agents can be stopped in the same way.

dikshant2210 commented 2 months ago

Thank you for the answer, I was able to run it without logging sensor data. I am running it on 2 GPU setup with one for Carla and one for running the model, but its still quite slow, it took roughly 14 hours for 16 scenarios (16 out of 220), is this to be expected or there is some problem?

jayyoung0802 commented 2 months ago

You can check the average time for each step in output logger.
Below is a rough estimate of each algorithm in the closed loop, (in H800)