search

zhejz / carla-roach

Roach: End-to-End Urban Driving by Imitating a Reinforcement Learning Coach. ICCV 2021.
https://zhejz.github.io/roach
Other
274 stars 50 forks source link

Adding Leaderboard Scenarios #8

Closed penghao-wu closed 2 years ago

penghao-wu commented 2 years ago

Hi, I wander have you tried to add the scenarios in Carla's scenario manager to your enviroment? I plan to do so but I am not sure whether py_trees used in the scenario manager supports multi-processing trainning (SubprocVecEnv). Thank you in advance!

zhejz commented 2 years ago

I spent some time trying to integrate the scenario_runner into the multi-processing RL training but it didn't work out smoothly. That's more than a year ago so I can't recall what problem exactly I've encountered. I remember my solution at that time was to write my own scenario_runner without using any dependencies (such as py_trees), but I didn't consider it as a priority.

hli2020 commented 2 years ago

Just out of curiosity, are you planning to test on the Carla sensor track leaderboard?

zhejz commented 2 years ago

Just out of curiosity, are you planning to test on the Carla sensor track leaderboard?

No, for that a scenario_runner has to be implemented or integrated, then all models have to be trained with those scenarios. Besides, to obtain a good ranking on the online leaderboard it is inevitable to spend time overfitting the secret testing env, which, in my opinion, does not add too much value.

Kin-Zhang commented 2 years ago

Hi, I wander have you tried to add the scenarios in Carla's scenario manager to your enviroment? I plan to do so but I am not sure whether py_trees used in the scenario manager supports multi-processing trainning (SubprocVecEnv). Thank you in advance!

I also want to run this Roach agent to see whether it can run at scenario manager the official branch leaderboard from carla, but it seems hard to extract the author's agent. lol