search

zhejz / carla-roach

Roach: End-to-End Urban Driving by Imitating a Reinforcement Learning Coach. ICCV 2021.
https://zhejz.github.io/roach
Other
293 stars 49 forks source link

Performance of Experts #16

Closed dq0803 closed 2 years ago

dq0803 commented 2 years ago

Hi

I'm trying to benchmark the RL experts 'iccv21-roach/trained-models/1929isj0: Roach' on w&b, but I couldn't achieve the results in the paper. The test suites is nocrash_dense, and the success rate is less than 0.5.

Does the trained model on w&b correspond to the test results in the paper？How many steps does it need to train to get the results in the paper？

zhejz commented 2 years ago

Does the trained model on w&b correspond to the test results in the paper?

Yes the trained model corresponds to the results in the paper. Make sure you are using the correct CARLA version because otherwise the performance will drop.

How many steps does it need to train to get the results in the paper?

1.7M steps in each of the 6 CARLA servers, that is 10M steps in total. Please refer to Section 4.1 of the paper for more details.