search

autonomousvision / carla_garage

[ICCV'23] Hidden Biases of End-to-End Driving Models
MIT License
203 stars 16 forks source link

training routes for s1 and s3 are same? #38

Closed buaazeus closed 2 months ago

buaazeus commented 2 months ago

Hello, I noticed that route waypoint and scenarios transform are same for s1 and s3, s1 is control loss, s3 is right turn, could you please explain why? Thank you.

carla_garage/leaderboard/data/training/routes/s1/Town01_Scenario1.xml carla_garage/leaderboard/data/training/routes/s3/Town01_Scenario3.xml

carla_garage/leaderboard/data/training/scenarios/s1/Town01_Scenario1.json carla_garage/leaderboard/data/training/scenarios/s3/Town01_Scenario3.json

Kait0 commented 2 months ago

S1 is control loss (rubble on the road). S3 is the pedestrian crossing scenario. Both of them can be placed at basically any location, so it is probably not an issue that they occur at the same location.

buaazeus commented 2 months ago

OK, Got it. The Scenario order are different from leaderboard.carla. Thank you.

NUMBER_CLASS_TRANSLATION = { "Scenario1": ControlLoss, "Scenario2": FollowLeadingVehicle, "Scenario3": DynamicObjectCrossing, "Scenario4": VehicleTurningRoute, "Scenario5": OtherLeadingVehicle, "Scenario6": ManeuverOppositeDirection, "Scenario7": SignalJunctionCrossingRoute, "Scenario8": SignalJunctionCrossingRoute, "Scenario9": SignalJunctionCrossingRoute, "Scenario10": NoSignalJunctionCrossingRoute }

https://leaderboard.carla.org/scenarios/

Kait0 commented 2 months ago

The scenario description on the website have changed since this paper. This is the description of the leaderboard 2.0 scenarios. I am not sure if the old leaderboard 1.0 scenario descriptions are still available.