Carla RSS compatibility with leaderboard and scenario runner

[AviBen96]

Good afternoon,

I have been experimented with Carla and scenario runner for a little while. I wanted to include the RSS sensor to one of my files so I switched to the RSS version. Dowloading and buulding the RSS API got me some errors so I simply downloaded the RSS zip. It worked fine with manual control.

Now I try to run my file with leaderboard and scenario runner and it fails. I have tried to find something link but I find very few info on Carla RSS. Any tip ?

Leaderboard fails to launch and get me this error.

pygame 2.1.3 (SDL 2.0.22, Python 3.8.10)
Hello from the pygame community. https://www.pygame.org/contribute.html

========= Preparing RouteScenario_0 (repetition 0) =========
> Loading the world

The scenario could not be loaded:

Traceback (most recent call last):
  File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/leaderboard/leaderboard_evaluator.py", line 227, in _load_and_run_scenario
    self._load_and_wait_for_world(args, config.town)
  File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/leaderboard/leaderboard_evaluator.py", line 171, in _load_and_wait_for_world
    self.world = self.client.load_world(town)
RuntimeError: map not found

> Registering the route statistics
Traceback (most recent call last):
  File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/leaderboard/leaderboard_evaluator.py", line 227, in _load_and_run_scenario
    self._load_and_wait_for_world(args, config.town)
  File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/leaderboard/leaderboard_evaluator.py", line 171, in _load_and_wait_for_world
    self.world = self.client.load_world(town)
RuntimeError: map not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/leaderboard/leaderboard_evaluator.py", line 441, in <module>
    main()
  File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/leaderboard/leaderboard_evaluator.py", line 435, in main
    leaderboard_evaluator.run(arguments)
  File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/leaderboard/leaderboard_evaluator.py", line 369, in run
    crashed = self._load_and_run_scenario(args, config)
  File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/leaderboard/leaderboard_evaluator.py", line 241, in _load_and_run_scenario
    self._cleanup()
  File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/leaderboard/leaderboard_evaluator.py", line 155, in _cleanup
    self.world.tick()  # TODO: Make sure all scenario actors have been destroyed
AttributeError: 'LeaderboardEvaluator' object has no attribute 'world'

Scenario Runner launches but crashes as soon as the agents tries to run (here the opposite vehicles tries to move). 

WARNING: Actor model vehicle.* not available. Using instead vehicle.tesla.model3
ScenarioManager: Running scenario OppositeVehicleJunction
Traceback (most recent call last):
  File "scenario_runner.py", line 415, in _load_and_run_scenario
    self.manager.run_scenario()
  File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/scenario_runner/srunner/scenariomanager/scenario_manager.py", line 136, in run_scenario
    self._tick_scenario(timestamp)
  File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/scenario_runner/srunner/scenariomanager/scenario_manager.py", line 175, in _tick_scenario
    self.scenario_tree.tick_once()
  File "/home/avner/.local/lib/python3.8/site-packages/py_trees/behaviour.py", line 158, in tick_once
    for unused in self.tick():
  File "/home/avner/.local/lib/python3.8/site-packages/py_trees/composites.py", line 578, in tick
    for node in child.tick():
  File "/home/avner/.local/lib/python3.8/site-packages/py_trees/composites.py", line 494, in tick
    for node in child.tick():
  File "/home/avner/.local/lib/python3.8/site-packages/py_trees/composites.py", line 494, in tick
    for node in child.tick():
  File "/home/avner/.local/lib/python3.8/site-packages/py_trees/composites.py", line 578, in tick
    for node in child.tick():
  File "/home/avner/.local/lib/python3.8/site-packages/py_trees/composites.py", line 494, in tick
    for node in child.tick():
  File "/home/avner/.local/lib/python3.8/site-packages/py_trees/composites.py", line 578, in tick
    for node in child.tick():
  File "/home/avner/.local/lib/python3.8/site-packages/py_trees/behaviour.py", line 247, in tick
    self.initialise()
  File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/scenario_runner/srunner/scenariomanager/scenarioatomics/atomic_behaviors.py", line 2059, in initialise
    self._agent = ConstantVelocityAgent(
  File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/PythonAPI/carla/agents/navigation/constant_velocity_agent.py", line 35, in __init__
    super().__init__(vehicle, target_speed, opt_dict=opt_dict, map_inst=map_inst, grp_inst=grp_inst)
TypeError: __init__() got an unexpected keyword argument 'map_inst'
__init__() got an unexpected keyword argument 'map_inst'
Destroying ego vehicle 109
ERROR: failed to destroy actor 109 : unable to destroy actor: not found 
Preparing scenario: OppositeVehicleRunningRedLight_2
WARNING: Actor model vehicle.* not available. Using instead vehicle.tesla.model3

CARLA version: 9.13 Platform/OS: Ubuntu 20.04

Steps to reproduce:

Download RSS zip, leaderboard ans scenario runner. Run any simulation.

Other information (documentation you consulted, workarounds you tried): Everything works fine with the normal carla version. The problems lays somewher in the Carla RSS API / Installation

[AviBen96]

There seems to be some problem with the map in my Carla RSS folder, but I haven't touched anything, the manual control works fine and even the bridge with autoware works normally (same problems as before let's say ^).

Any tips ?

Thank you very much !!

[glopezdiest]

Firstly, I edited your comment for better readability of the code. Secondly, as you said, you seem to have an issue with the map, the rest of the scenario runner and leaderboard issues all stem from that one.

the manual control works fine and even the bridge with autoware works normally (same problems as before let's say ^)

Not sure what you mean here. Do you mean that the CARLA RSS works fine when not using the Leaderboard? Can you load a map using CARLA RSS without the leaderboard?

[AviBen96]

Yes I can launch Carla simulator properly, run the manual control and even the manual control with rss.

Only scenario runner and leaderboard seem to crash. Is there some maps missing in the Carla_RSS zip compared to the normal one ? If so how can I fix this ? I am asking this because in HDMaps there is the maps 1 to 7 but I do not know where are map 10 and 12 (and I think this is the problem for leaderboard).

[AviBen96]

Thank you for the edit !

[glopezdiest]

How are you running the leaderboard exactly? And no, as far as I know, the maps are the same in both CARLA versions

[AviBen96]

I did exactly the same as what I did with Carla not RSS :

Here is a screenshot of my bashrc and my file structure. Since I have both carla autoware bridge and leaderboard running in the same machine, I have the path of the bridge in my bashrc and the export in the run file (it was working properly before RSS, so this is not the problem).

I have installed leaderboard inside the Caral_RSS folder and scenarioRunner in the leaderboard folder - again same as before.

I truly think the problem lays somewhere in the Carla_RSS zip that must have some files differences. There was one agent missing in the carla Python folder too that I added from the other zip. I think some map files are missing in the RSS zip, but I am afraid to break everything if I start mixing the regular carla and the carla rss folder ...

[AviBen96]

Any tip here ? I truly don't know what to do. I built a script that seems to work properly but relised on RSS. If I can't run leaderboard nor scenrio runner it is pretty much useless ...

[glopezdiest]

What map is giving you issues exactly? Can it be that you haven't downloaded it from the AdditionalMaps package?

[AviBen96]

I will try again but I am pretty sure I did Import the Additional Maps.

I think it is a deeper problem. Leaderboard doesn’t launch because of the map 12 that is missing. But scenarioRunner crashes on every map, and it does launch so it has access to the map. It crashes when the second actor starts moving (here the opposite vehicle on red light). I truly don’t understand what is missing.

One file from the Python API was missing too in the Carla RSS zip.

I will try to make rss library in my regular Carla installation since all the rest is working there and see if it breaks the existing installation.

[glopezdiest]

Okay. I've never used CARLA RSS before, but I do use the Leaderboard quite a lot, so I was trying to make sure that it wasn't some kind of missconfiguration. I'll look into it, but I can't promise any results right now.

[AviBen96]

Thank you very much for your help !

No I truly don’t think it is missconfiguration, I have been using leaderboard for a few months too and it was working properly.

I will let you know too if I make any progress in the matter.

Thank you again !

[AviBen96]

I don't manage to build the RSS library on my original carla folder.

The RSS still works fine, but it can't access the maps properly for some reason.

I truly think it is a problem in the maps access in the Zip of Carla RSS.

I have the autoware bridge with its prioritary maps that runs just fine and leaderboard and scenario runner runs fine.

In the carla_RSS map 12 is missing, and some access to the map causes the crash of scenario runner.

I think it is a small map problem that can be easily fixed by someone who understands the ways maps are accessed by leaderboard and scenario runner.

Any tip on this ? I am trying to compare my original folder and the RSS one to get what missing, but would love some tips / help !

Thanks

[AviBen96]

Copy pasting the folder map12 of Content /carla/map I now get this error

WARNING: Traffic sign 36783 overlaps a driving lane. Moving out of the road... WARNING: Failed to find suitable place for signal. WARNING: Traffic sign 37270 overlaps a driving lane. Moving out of the road... WARNING: Failed to find suitable place for signal. WARNING: Traffic sign 36066 overlaps a driving lane. Moving out of the road... WARNING: Failed to find suitable place for signal. WARNING: Traffic sign 36672 overlaps a driving lane. Moving out of the road... WARNING: Failed to find suitable place for signal. WARNING: Traffic sign 35879 overlaps a driving lane. Moving out of the road... WARNING: Failed to find suitable place for signal. WARNING: Traffic sign 35773 overlaps a driving lane. Moving out of the road... WARNING: Traffic sign 36157 overlaps a driving lane. Moving out of the road... WARNING: Failed to find suitable place for signal. WARNING: Traffic sign 35956 overlaps a driving lane. Moving out of the road... WARNING: Failed to find suitable place for signal. WARNING: Traffic sign 36204 overlaps a driving lane. Moving out of the road... WARNING: Failed to find suitable place for signal. WARNING: Traffic sign 36070 overlaps a driving lane. Moving out of the road... WARNING: Failed to find suitable place for signal. WARNING: Traffic sign 37020 overlaps a driving lane. Moving out of the road... WARNING: Failed to find suitable place for signal. WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle.* not available. Using instead vehicle.tesla.model3 Skipping scenario 'SignalizedJunctionLeftTurn_1' due to setup error: Found no traffic lights, use the non signalized version instead

Traceback (most recent call last): File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/leaderboard/scenarios/route_scenario.py", line 288, in _build_scenarios scenario_instance = scenario_class(world, [ego_vehicle], scenario_config, timeout=timeout) File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/scenario_runner/srunner/scenarios/signalized_junction_left_turn.py", line 164, in init super().init(world, ego_vehicles, config, randomize, debug_mode, criteria_enable, timeout) File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/scenario_runner/srunner/scenarios/signalized_junction_left_turn.py", line 78, in init super().init("JunctionLeftTurn", File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/scenario_runner/srunner/scenarios/basic_scenario.py", line 61, in init self._initialize_actors(config) File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/scenario_runner/srunner/scenarios/signalized_junction_left_turn.py", line 175, in _initialize_actors raise ValueError("Found no traffic lights, use the non signalized version instead") ValueError: Found no traffic lights, use the non signalized version instead

WARNING: Actor model static.prop.warningaccident not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model static.prop.warningconstruction not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model static.prop.warningaccident not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model static.prop.warningconstruction not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model static.prop.warningaccident not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model static.prop.warningconstruction not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle.. not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle.. not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model static.prop.warningconstruction not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle.. not available. Using instead vehicle.tesla.model3 WARNING: Actor model static.prop.warningaccident not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model static.prop.busstoplb not available. Using instead vehicle.tesla.model3 WARNING: Actor model static.prop.warningconstruction not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle.* not available. Using instead vehicle.tesla.model3 Skipping scenario 'PriorityAtJunction_1' due to setup error: No traffic lights found, nothing to do here

Traceback (most recent call last): File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/leaderboard/scenarios/route_scenario.py", line 288, in _build_scenarios scenario_instance = scenario_class(world, [ego_vehicle], scenario_config, timeout=timeout) File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/scenario_runner/srunner/scenarios/green_traffic_light.py", line 40, in init super().init("PriorityAtJunction", File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/scenario_runner/srunner/scenarios/basic_scenario.py", line 61, in init self._initialize_actors(config) File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/scenario_runner/srunner/scenarios/green_traffic_light.py", line 65, in _initialize_actors self._get_traffic_lights(self._junction, ego_junction_dist) File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/scenario_runner/srunner/scenarios/green_traffic_light.py", line 71, in _get_traffic_lights raise ValueError("No traffic lights found, nothing to do here") ValueError: No traffic lights found, nothing to do here

WARNING: Actor model static.prop.warningaccident not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model static.prop.foodcart not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle. not available. Using instead vehicle.tesla.model3 WARNING: Actor model vehicle.. not available. Using instead vehicle.tesla.model3 WARNING: Actor model static.prop.warningconstruction not available. Using instead vehicle.tesla.model3

The scenario could not be loaded:

Traceback (most recent call last): File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/leaderboard/leaderboard_evaluator.py", line 228, in _load_and_run_scenario scenario = RouteScenario(world=self.world, config=config, debug_mode=args.debug) File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/leaderboard/scenarios/route_scenario.py", line 80, in init self._spawn_parked_vehicles() File "/home/avner/CARLA_AUTOWARE_project/carla_RSS/leaderboard/leaderboard/scenarios/route_scenario.py", line 180, in _spawn_parked_vehicles mesh_bp.set_attribute("mesh_path", mesh_path) IndexError: attribute 'mesh_path' not found

Registering the route statistics Registering the global statistics

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.