Autoware.auto / LGSVL simulator - AVP demo: Connection problems - crash: lgsvl_simulation_main.py: agent is missing bridge client

[filipthoen2]

Hello there!

I'm facing some connection issues between Autoware.auto / LGSVL simulator when trying to run the Autoware.auto AVP demo. I'm using Ubuntu 18.04, and using 'ade' (so all prebuild containers).

When running the last command of the following sequence (Terminal 2): ade enter ade$ source /opt/AutowareAuto/setup.bash ade$ ros2 launch autoware_demos avp_sim.launch.py

I get an exception causing one of the processes to die:

[lgsvl_simulation_main.py-10] Traceback (most recent call last):
[lgsvl_simulation_main.py-10]   File "/opt/AutowareAuto/lib/lgsvl_simulation/lgsvl_simulation_main.py", line 38, in <module>
[lgsvl_simulation_main.py-10]     main()
[lgsvl_simulation_main.py-10]   File "/opt/AutowareAuto/lib/lgsvl_simulation/lgsvl_simulation_main.py", line 27, in main
[lgsvl_simulation_main.py-10]     node.setup()
[lgsvl_simulation_main.py-10]   File "/opt/AutowareAuto/lib/python3.8/site-packages/lgsvl_simulation/lgsvl_simulation_node.py", line 74, in setup
[lgsvl_simulation_main.py-10]     if not self.vehicle_agent.bridge_connected:
[lgsvl_simulation_main.py-10]   File "/opt/AutowareAuto/lib/python3.8/site-packages/lgsvl/agent.py", line 169, in bridge_connected
[lgsvl_simulation_main.py-10]     return self.remote.command("vehicle/bridge/connected", {"uid": self.uid})
[lgsvl_simulation_main.py-10]   File "/opt/AutowareAuto/lib/python3.8/site-packages/lgsvl/remote.py", line 69, in command
[lgsvl_simulation_main.py-10]     raise Exception(data["error"])
[lgsvl_simulation_main.py-10] Exception: Agent '49a2549e-4f16-4996-a3bd-9f825c6fd695' is missing bridge client
[ERROR] [lgsvl_simulation_main.py-10]: process has died [pid 825, exit code 1, cmd '/opt/AutowareAuto/lib/lgsvl_simulation/lgsvl_simulation_main.py --ros-args -r __node:=lgsvl_simulation_node -r __ns:=/drivers --params-file /opt/AutowareAuto/share/autoware_demos/param/avp/lgsvl_simulation.param.yaml --params-file /tmp/launch_params_72ndpl7_'].

Any tips? Full detailed steps & more questions below.

Thanks a bunch, Filip

----------------------- Detailed steps below ----------------------------------

Terminal 1: running LGSVLsimulator & Rviz visualization (part of ROS)

1. Execute: export ADE_HOME=/home/demo/adehome/ cd ~/AutowareAuto ade --rc .aderc-amd64-foxy-lgsvl start --update --enter ade$ /opt/lgsvl/simulator &

   2. The SVL Simulator will come up. a. Click 'Open Browser'. b. Log into the LGSVL web site c. make a new simulation (see detailed instructions at: https://autowarefoundation.gitlab.io/autoware.auto/AutowareAuto/lgsvl.html) or choose an exiting one d. the top widget on the LGSVL simalator window will change & now should say "API ready!"

   3. Next start the visualization:

   ade$ source /opt/AutowareAuto/setup.bash ade$ ros2 launch autoware_auto_launch autoware_auto_visualization.launch.py

   Result:

   • LGSVLSimulator windows will still show its empty grey window, with 'API Ready!' msg
   • The Rviz visualization window will open showing a grid.

   • The terminal output will show a running list of warnings - see Troubleshooting (https://autowarefoundation.gitlab.io/autoware.auto/AutowareAuto/avpdemo.html#avpdemo-simulation-troubleshooting)

      ...
      Warning: invalid frame ID "map" passed to CanTransport argument target_frame - frame does not exist
         at line 133 in /tmo/binarydeb/ros-foxy-tf2-0.13.13/src/buffer_core.cpp
     
      and then stops with:
     
       [rviz2-1] Parsing robot urdf xml string.
       [rviz2-1] Parsing robot urdf xml string.

   Note: - this will start following processes: rviz2-1, rosbridge_websocket (port 9090) & Python web server (port 8000)

   • The web server (http://localhost:8000/) provides a simple graphical web interface to park or bring back the car (emulating a AVP mobile phone app).
   • rosbridge_websocket is a ROS2 component (/opt/ros/foxy/lib/rosbridge_server/rosbridge_websocket) & creates a web socket on port 9090. It allows to publish/subscribe to ROS topics over a web socket.

Terminal 2: run AVP demo

1. open new terminal

2. Execute:

   ade enter ade$ source /opt/AutowareAuto/setup.bash ade$ ros2 launch autoware_demos avp_sim.launch.py

   Result:

   • LGSVLSimulator windows will show a Lexus SUV car.
   • Rviz will now show a map of the AVP parking lot.

   I get an exception causing one of the processes to die:

   [lgsvl_simulation_main.py-10] Traceback (most recent call last): [lgsvl_simulation_main.py-10] File "/opt/AutowareAuto/lib/lgsvl_simulation/lgsvl_simulation_main.py", line 38, in [lgsvl_simulation_main.py-10] main() [lgsvl_simulation_main.py-10] File "/opt/AutowareAuto/lib/lgsvl_simulation/lgsvl_simulation_main.py", line 27, in main [lgsvl_simulation_main.py-10] node.setup() [lgsvl_simulation_main.py-10] File "/opt/AutowareAuto/lib/python3.8/site-packages/lgsvl_simulation/lgsvl_simulation_node.py", line 74, in setup [lgsvl_simulation_main.py-10] if not self.vehicle_agent.bridge_connected: [lgsvl_simulation_main.py-10] File "/opt/AutowareAuto/lib/python3.8/site-packages/lgsvl/agent.py", line 169, in bridge_connected [lgsvl_simulation_main.py-10] return self.remote.command("vehicle/bridge/connected", {"uid": self.uid}) [lgsvl_simulation_main.py-10] File "/opt/AutowareAuto/lib/python3.8/site-packages/lgsvl/remote.py", line 69, in command [lgsvl_simulation_main.py-10] raise Exception(data["error"]) [lgsvl_simulation_main.py-10] Exception: Agent '49a2549e-4f16-4996-a3bd-9f825c6fd695' is missing bridge client [ERROR] [lgsvl_simulation_main.py-10]: process has died [pid 825, exit code 1, cmd '/opt/AutowareAuto/lib/lgsvl_simulation/lgsvl_simulation_main.py --ros-args -r node:=lgsvl_simulation_node -r ns:=/drivers --params-file /opt/AutowareAuto/share/autoware_demos/param/avp/lgsvl_simulation.param.yaml --params-file /tmp/launch_params72ndpl7'].

   Notes:

   • in the car configuration in WISE web interface, the 'ROS2ForUnitySVLBridge' is a ROS2 bridge implementation based on ros2-for-unity project: see https://github.com/RobotecAI/ros2-for-unity. You are required to add this to your vehicle.
   • the self.remote.command("vehicle/bridge/connected" Python cmd will send the 'vehicle/bridge/connected' data via a web socket. I suspect the 'bridge client' mentioned in the crash output is the one of SVL simulator, which is trying to connect to the web socket on port 9090 of ROS2 'rosbridge_websocket' (see above) & trying to publish the 'vehicle/bridge/connected' ROS2 topic. So LGSVL is not starting its web client?
   • 'config.yml' in top-level folder of SVL simulator defines ports 8080 (?) & 8181 (api_port). Only port 8181 seems connectable/up. The api_port is used by Python scripts to control the SVL simulator.

   However, the output will continue after the crash & show the following repeating message continuously:

   [freespace_planner_node_exe-20] [INFO] [1673572942.785926406] [planning.freespace_planner]: Waiting for costmap generator action service... [behavior_planner_node_exe-22] [INFO] [1673572943.111469042] [planning.behavior_planner_node]: Waiting for action server 'plan_parking_trajectory'... [costmap_generator_node_exe-19] [INFO] [1673572943.208381560] [planning.costmap_generator_node]: Waiting for initial pose...

   I tried to set the 'initial pose' for the NDT, but the 'waiting for initial pose' message keeps being printed, likely as the Rviz can't communicate with SVL Simulator due to lack of web client on its end.

   One thing I'm wondering is how my sensor configuration is being used in the 'AWFLexus2016RXHybrid' vehicle. When you use the WISE webinterface to click on this vehicle in your Library, it shows about ~20 sensor configurations. I don't see a way to pick a specific sensor configuration when I create an new simulation in the same WISE web interface. Am I missing something?

   Looks like '/opt/AutowareAuto/share/autoware_demos/param/avp/lgsvl_simulation.param.yaml' configures 'scene_uuid' & 'config_uuid' (= AWFLexus2016RXHybrid Autoware.Auto configuration) (as well as vehicle spawn location).

   /opt/AutowareAuto/share/autoware_demos/launch/avp_sim.launch.py loads teh different ROS2 nodes, making up the AVP application.

   I checked 'lgsvl_simulation.param.yaml' which sets:

   config_uuid: c7d50f84-0a6b-4007-9b34-a723dc0e3d20

   I tediously checked all the IDs of all sensor configurations in WISE for the 'AWFLexus2016RXHybrid' -- I didn't find a match! So is this reason for the lack of bridge client?

[Sam827-r]

Hi @filipthoen2 Can you provide demo video for your issue.

[filipthoen2]

Sam827-r - thanks for offering to help!

Here is a screen recording (YouTube as too big for github):

https://youtu.be/XCtoNNOrNlM

One thing I was wondering is rgd the LGSVL Simulator setup: on the setup page (https://autowarefoundation.gitlab.io/autoware.auto/AutowareAuto/lgsvl.html), it is not clear whether you have to run the steps of the "Bridging with Autoware.Auto" section (towards the end). Is this needed?

Thanks again,

Filip

[filipthoen2]

Anybody any further suggestions on how to get this connection issue resolved?

Thanks much,

Filip

[Sam827-r]

Hi@ Filip, It's seems to be you mess up with bridge plugins could you please check it along with sensor configuration, so api gets connection.

[filipthoen2]

Thanks, Sam827-r.

Two things which are not clear to me are the following:

1. When adding my sensor configuration to the 'AWFLexus2016RXHybrid' vehicle in the WISE web interface, I don't understand how my configuration is actually being used during a simulation (when drilling into the 'AWFLexus2016RXHybrid' vehicle I see 20+ predefined configurations).

Note my vehicle sensor configuration has the 'ROS2ForUnitySVLBridge' bridge attached. I would assume this is sufficient to connect to Autoware.auto stack - do I need to so something extra?

2. When defining a simulation in the WISE web interface, there is no way of selecting which vehicle is actually being used. How is this selection then realized?

Thanks,

Filip

[Sam827-r]

Try simulation in Random traffic template you will get know all points.

[filipthoen2]

@Sam827-r I created a simulation in the WISE web interface to run random traffic. During the creation steps, you indeed get to chose all the different options (map, vehicle, sensorconfig, etc).

However I just want to run the Autonomous Parking Valet (AVP) demo -- as described here: https://autowarefoundation.gitlab.io/autoware.auto/AutowareAuto/avpdemo.html.

From the described steps, this is an "API only" type of simulation, and this is the type you need to select in the WISE web interface (this type of sim will not give you the options to select map, veh, sensor config, etc.). I understood that in this case these settings are programmatically controlled via the launching 'avp_sim.launch.py ' script.

This script will load the following 2 param files:

1. param/avp/lgsvl_interface.param.yaml: this defines the vehicle control commands ('ackermann' cmd)
2. param/avp/lgsvl_simulation.param.yaml: the sets the scene_uuid & (vehicle) 'config_uuid':

/**: ros__parameters:

AutonomouStuff scene

scene_uuid: 2aae5d39-a11c-4516-87c4-cdc9ca784551
vehicle:
  # AWFLexus2016RXHybrid Autoware.Auto configuration
  config_uuid: c7d50f84-0a6b-4007-9b34-a723dc0e3d20
  # Vehicle spawn position
  spawn:
    position:
      x: -108.094
      y: -2.0545
      z: -28.905
    rotation:
      x: 1.641
      y: 141.217
      z: 359.085

I traced '2aae5d39-a11c-4516-87c4-cdc9ca784551' scene_uuid back & it indeed is the 'AutonomouStuff' map added from the asset Store in the WISE web interface - good! However, I checked all sensor configurations of the "AWFLexus2016RXHybrid" vehicle & can't find the above 'c7d50f84-0a6b-4007-9b34-a723dc0e3d20' number.

Before I launched the ROS components ('ros2 launch autoware_demos avp_sim.launch.py'), I checked whether this terminal can connect to the Autoware.auto bridge (as launched from the 1st terminal) by using the 'telnet localhost 9090' cmd -- this works fine. SO the Autoware bridge is up, I guess my vehicle doesn't spawn a 'ROS2ForUnitySVLBridge' client on its end?

Any tips?

Filip

[Sam827-r]

Hi @filipthoen2

Just for testing cause check sim is connected to bridge and data transfer happening on bridge

[filipthoen2]

@Sam827-r -- I've figured this out, it was the 'config_uuid' specified in the 'lgsvl_simulation.param.yaml'. This ID doesn't seem to exist in the WISE web interface anymore -- I replaced it with my sensor configuration, and this solved the issue!

Thanks for all your help!

Filip

[filipthoen2]

@Sam827-r -- I've figured this out, it was the 'config_uuid' specified in the 'lgsvl_simulation.param.yaml'. This ID doesn't seem to exist in the WISE web interface anymore -- I replaced it with my sensor configuration, and this solved the issue!

Thanks for all your help!

Filip