search

zhejz / carla-roach

Roach: End-to-End Urban Driving by Imitating a Reinforcement Learning Coach. ICCV 2021.
https://zhejz.github.io/roach
Other
274 stars 50 forks source link

Metric #14

Closed neilsambhu closed 2 years ago

neilsambhu commented 2 years ago

In ego_vehicle_handler.py, is "score_route" "success rate"? I know "score_composed" is the "driving score".

neilsambhu commented 2 years ago

@zhejz, #14 is an unanswered question.

zhejz commented 2 years ago

"score_route" is the percentage of route completed, with respect to one route. "score_composed" is computed by multiplying "score_route" with the "score_penalty", these metrics are used by the carla leaderboard. "success_rate" is the percentage of routes that are fully completed, with respect to many routes. This metric is used by the nocrash benchmark. You can find more details about the metrics in the paper and the website of the carla leaderboard.

neilsambhu commented 2 years ago

@zhejz, thank you for your response.