Import Issues with Carla Simulator 0.9.13 release

[marcusvinicius178]

Hi @hatem-darweesh

I am following your tutorial: Autoware Universe with CARLA My OS is ubuntu 20.04 and I have CARLA 0.9.13 version installed at:

I am having issues to run the /run_exploration_mode_ros2.sh

I receive the following import errors:

File "hatem-repos/op_bridge/op_bridge/op_ros2_agent.py", line 35, in <module>
    from sensor_msgs_py.point_cloud2 import create_cloud
ModuleNotFoundError: No module named 'sensor_msgs_py'

It is not an egg or python version issue, because I have checked that other modules are imported...

However I checked other thing that I would like to ask you.: I needed to change the script **/run_exploration_mode_ros2.sh*, to be able to execute the script op_ros2_agent.py**. Please take a look at the end rows:

I was inspecting the ${LEADERBOARD_ROOT} path in sublime text, and I got:

Going straight to the point:

The ${LEADERBOARD_ROOT} points to: /home/user/carla-0.9.13/op_bridge

However my carla-simulator is installed at opt/carla-simulator I have not this carla-0.9.13 folder. Actually I have dowloaded it from here: https://github.com/carla-simulator/carla/tree/0.9.13 Afterwards I have uncompressed it. And checked the content:

I do not have the content contained in the export env variables:

export CARLA_ROOT=/home/user/carla-0.9.13/CARLA_0.9.13_RSS
   12  export SCENARIO_RUNNER_ROOT=/home/user/carla-0.9.13/scenario_runner
   13: export LEADERBOARD_ROOT=/home/user/carla-0.9.13/op_bridge
   14  export TEAM_CODE_ROOT=/home/user/carla-0.9.13/op_agent
   15  export PYTHONPATH=$PYTHONPATH:${CARLA_ROOT}/PythonAPI

I have not found inside "carla-0.9.13" CARLA_0_9_13_RSS or scenario_runner or op_bridge or op_agent. Just Python_API is available. The other folder I have put inside "hatem-repos" folder as you recommend in your tutorial:

The folder CARLA_0_9_13_RSS I have no idea where it is.

Then my question is simple, how did you install the CARLA 0.9.13 simulator and could I take a look how your directories are organized please?

I do not know if commenting the line:

python3 ${LEADERBOARD_ROOT}/op_bridge/op_bridge_ros2.py

and replacing for python3 hatem-repos/op_bridge/op_bridge/op_ros2_agent.py

is generating my issue. Becuse I guee the ${LEADERBOARD ROOT} term accomplishes with some function in CARLA.

Would you have some idea what is going on?

Thanks in advance

[hatem-darweesh]

I downloaded CARLA from github (the release section) https://github.com/carla-simulator/carla/releases/tag/0.9.13

with or without RSS are the same

the folder structure:

carla-0.9.13

• CARLA_0.9.13_RSS
• op_agent
• op_bridge
• scenario_runner

[marcusvinicius178]

Hi Mr Darweesh thanks for attention!

Ok now I was able to do pip install -r requirements.txt in each of these folder.

However I am struggling to run the sh file inside op_scripts folder. I have overcame one of the errors:

" No module srunner...." Just issuing pip install srunner

All the common modules are being imported successfully: Import math Import Tf....etc

However it is failing to import any module from leaderboard ( And I have already installed the requirements.txt with pip in leaderboard folder)

" From leaderboard import ...."

I tried a lot of solutions to "import python" modules over the internet, giving the absolute path to the modules, etc. But anyone worked.

Just tell me did you need to change the import statements in order to get the bridge working? Or do some additional installation? I am using python 3.8 and 3.7 egg file. I tried the 2.7 python version and 2.7 egg file (sourced in bash export statement). However I had no success.

Suggest some workaround? Thanks in advance

[marcusvinicius178]

@hatem-darweesh The error after execute the file python3 op_bridge_ros2.py is

`Traceback (most recent call last):
  File "op_bridge_ros2.py", line 16, in <module>
    from op_bridge.agent_wrapper import AgentWrapper
  File "/home/rota2030/hatem-repos/op_bridge/op_bridge/op_bridge.py", line 15, in <module>
    from leaderboard.autoagents.agent_wrapper import  AgentWrapper
ModuleNotFoundError: No module named 'leaderboard.autoagents'`

This module cannot be imported anyway I have already removed the leadeboard folder inside the op_bridge folder. Then cloned it in hatem-repos folder and done the pip install -r leaderboard/requirements.txt...

I have changed my bash file to show the PythonPath

`#!/bin/bash
export PYTHONPATH=$PYTHONPATH:/opt/carla-simulator/PythonAPI/carla/dist/carla-0.9.13-py3.7-linux-x86_64.egg
export PYTHONPATH=$PYTHONPATH:/opt/carla-simulator/PythonAPI/carla
export PYTHONPATH=$PYTHONPATH:/opt/carla-simulator/PythonAPI
export PYTHONPATH=$PYTHONPATH:${CARLA_ROOT}/PythonAPI/util
export PYTHONPATH=$PYTHONPATH:${CARLA_ROOT}/PythonAPI/carla
export PYTHONPATH=$PYTHONPATH:${CARLA_ROOT}/PythonAPI/carla/agents
#export PYTHONPATH=$PYTHONPATH:${LEADERBOARD_ROOT}/autoagents
#export PYTHONPATH=$PYTHONPATH:/home/user/hatem-repos/scenario_runner
export PYTHONPATH="${CARLA_ROOT}/PythonAPI/carla/":"${SCENARIO_RUNNER_ROOT}":"${LEADERBOARD_ROOT}":"${CARLA_ROOT}/PythonAPI/carla/dist/carla-0.9.10-py3.7-linux-x86_64.egg":${PYTHONPATH}

export CARLA_ROOT=/opt/carla-simulator
export LEADERBOARD_ROOT=/home/rota2030/hatem-repos/leaderboard
export TEAM_CODE_ROOT=/home/user/hatem-repos/op_agent
export SCENARIO_RUNNER_ROOT=/home/user/hatem-repos/scenario_runner

#source /opt/ros/galactic/setup.bash 
#I have uninstalled ros galactic and installed foxy. Because Leaderboard modules just work with ros2 foxy according to https://leaderboard.carla.org/get_started/
source ~/ros2_foxy/install/local_setup.bash
source /home/rota2030/hatem-repos/carla-autoware-universe/autoware.universe.openplanner/install/setup.bash`

However anything worked.

I tried a lot of approaches an you can check below to force Python see this leaderboard module:

`from srunner.scenariomanager.carla_data_provider import *
#from op_bridge.leaderboard.autoagents.agent_wrapper import  AgentWrapper
from op_bridge.agent_wrapper import AgentWrapper

"""
#Line below didn't work
#sys.path.append(os.path.join(os.path.abspath(os.path.dirname(__file__), '..','agent_wrapper.py'))
#import AgentWrapper

#import sys
#sys.path.append("/home/rota2030/hatem-repos/leaderboard/leaderboard/autoagents/")
#import AgentWrapper

#imp.load_source(AgentWrapper, 'home/rota2030/hatem-repos/leaderboard/leaderboard/autoagents/')

#LINE BELOW WORKED WITH FULL PATH
from importlib.machinery import SourceFileLoader

auto_agents = SourceFileLoader("agent_wrapper", "/home/rota2030/hatem-repos/leaderboard/leaderboard/autoagents/agent_wrapper.py").load_module() auto_agents.AgentWrapper() """`

Anyone worked.

[yuting-fu]

You need to export the leaderbord root before you use LEADERBOARD_ROOT all over the place. Make sure the line below is correctly pointing to your leaderboard installation, and make sure this line is above any other export related to Leaderboard root:

export LEADERBOARD_ROOT=/home/rota2030/hatem-repos/leaderboard

[yuting-fu]

Same with CARLA_ROOT and the other variables in your setup script. Make sure you have correctly configured them before using them.

[marcusvinicius178]

Hi @yuting-fu the directories and export expression are correct. Please take a look

However the error persisted. I followed your suggestion and used export LEADERBOARD_ROOT=/home/rota2030/hatem-repos/leaderboard before using it. However if you check, I did not use it later. It is commented where I tried to use previously. Teh scenario runner worked well, even if I comment the export SCENARIO_RUNNER line, it works normally.

[yuting-fu]

The error persists because you export leaderboard root but do not use it in your pythonpath. So it cannot find the leaderborad module. Scenario runner works because you had one line commented out, but there is another line export it below that is not commented out.

Please make sure each module path is correctly configured in the bash. In principe the script from Hatem works fine, you only need to update a couple of paths to adapt to your own setup. You don’t have to write your own script for that. If you do, make sure you include everything that’s required.

[yuting-fu]

You didn’t show the complete line number in your screenshot so it’s hard to refer to it.

But you configured the Carla .egg twice with different Carla versions. I guess you want to use 9.13.0, then you should remove the one with 0.9.10 egg a few lines below.

your bash is called new_export.sh, before you run the python3 command with the op_bridge_ros2.py, did you source your new_export.sh already? Make sure you do $ source new_export.sh Then run the python3 command.

[yuting-fu]

Check the last post from the other issue in this repo as well. Your bash environment is probably messed up a bit at this point. It will be helpful to first clean up your pythonpath and then start debugging again. Good luck.

[marcusvinicius178]

Hi @yuting-fu thanks for your hints. I really appreciate the assistance. Last points to check with you, and considerations

1 The exports statements don't work in a "special customized bash". I needed to do directly in terminal. Otherwise they don't change the path you export.

2The environment varible path which contains "user" could not be identified by my system. Maybe the correct be ${USER}, otherwise the modules will never be found. In my case I was just replacing the path for my username and so on...

I will let my case here if someone wishes as example:

3 After doing these adjusments the terminal runned the bridge script, however the CARLA simulator frozen and I received a runtime error. I would like to check with you @hatem-darweesh or @yuting-fu if this time error can be also a script error or if is related to hardware capabilties. Please take a look below

I am going to copy and paste the error below:

1668692448.254837 [0]    python3: using network interface wlp0s20f3 (udp/10.144.0.20) selected arbitrarily from: wlp0s20f3, docker0
Ego Vehicle:  hero
Executing stack... hero Town01
-------------------------------
Starting OpenPlanner .. 
hero
Town01
true

-------------------------------
/home/rota2030/hatem-repos/op_agent/start_ros2.sh: line 15: /home/hatem/hatem-repos/autoware-universe/autoware/install/setup.bash: No such file or directory
Traceback (most recent call last):
  File "/home/rota2030/hatem-repos/op_bridge/op_bridge/op_bridge_ros2.py", line 94, in _tick_agent
    ego_action = self.agent()
  File "/home/rota2030/hatem-repos/op_bridge/leaderboard/autoagents/agent_wrapper.py", line 75, in __call__
    return self._agent()
  File "/home/rota2030/hatem-repos/op_bridge/leaderboard/autoagents/autonomous_agent.py", line 115, in __call__
    control = self.run_step(input_data, timestamp)
  File "/home/rota2030/hatem-repos/op_bridge/op_bridge/op_ros2_agent.py", line 536, in run_step
    raise RuntimeError("Stack exited with: {} {}".format(
RuntimeError: Stack exited with: 1 None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/rota2030/hatem-repos/op_bridge/op_bridge/op_bridge_ros2.py", line 204, in run_agent
    self.agent_loop._tick_agent(timestamp)
  File "/home/rota2030/hatem-repos/op_bridge/op_bridge/op_bridge_ros2.py", line 100, in _tick_agent
    raise AgentError(e)
leaderboard.autoagents.agent_wrapper.AgentError: Stack exited with: 1 None
Scenario Ended , Hero has fallen, Sayonara
ERROR: failed to destroy actor 197 : unable to destroy actor: not found 
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 1388, in _shutdown
    lock.acquire()
TypeError: _stop_loop() takes 1 positional argument but 2 were given

For last. I just wish to say that I am running this simulation in a very powerful NUC with the following features:

Processor: intel i7 Memory: 64 GB GPU: NVIDIA RTX 2060 w/ 6GB VRAM included. memory: DDR4-3200 1.2V SO-DIMM Fequency clock: 4.70 GHz

The complete hardware specifications is here: https://www.intel.com/content/www/us/en/products/sku/202783/intel-nuc-11-enthusiast-kit-nuc11phki7c/specifications.html

**Would you know if this computer is not so strong enough to run CARLA + Autoware in the same machine?

Or this runtime error is related to other thing? (I don't know internet velocity on my university? or a code issue?)

Or should I have runned other module before? Set some argument or flag to run this python code? **

Thanks in advance

[xpharry]

Hi @marcusvinicius178

Your PC is quite enough strong for Carla and Autoware. And it has noting to do with your internet bandwidth, which answers your question in another channel. Carla simulator is playing as a server in your local computer.

I think you still have multiple things you can try. Basically you need to follow tightly the tutorial videos.

1. I guess your system is Ubuntu 20.04 since you are using ROS foxy. Then you need to change your carla_..egg to Python3.7 version.
2. In Dr. Hatem's tutorial videos, he mentioned some modifications in the scrips. Not sure if you have followed it. i. op_bridge/op_scripts/run_exploration_mode_ros2.sh ii. op_agent/start_ros2.sh
3. In the videos, the launch file and the sensor params also got modified. i. autoware.universe.openplanner/src/launcher/autoware_launch/autoware_launch/launch/autoware.launch.xml ii. autoware.universe.openplanner/src/sensor_kit/sample_sensor_kit_launch/sample_sensor_kit_launch/launch/gnss.launch.xml iii. autoware.universe.openplanner/src/sensor_kit/sample_sensor_kit_launch/sample_sensor_kit_description/config/sensor_kit_calibration.yaml iv. autoware.universe.openplanner/src/sensor_kit/sample_sensor_kit_launch/sample_sensor_kit_description/config/sensors_calibration.yaml

[marcusvinicius178]

Hi @xparry

Hi @marcusvinicius178

Your PC is quite enough strong for Carla and Autoware. And it has noting to do with your internet bandwidth, which answers your question in another channel. Carla simulator is playing as a server in your local computer. Nice to know thanks I think you still have multiple things you can try. Basically you need to follow tightly the tutorial videos.

1. I guess your system is Ubuntu 20.04 since you are using ROS foxy. Then you need to change your carla_..egg to Python3.7 version.

Ok I have changed, and exported the egg 3.7 env variable. take a look

2. In Dr. Hatem's tutorial videos, he mentioned some modifications in the scrips. Not sure if you have followed it. i. op_bridge/op_scripts/run_exploration_mode_ros2.sh ii. op_agent/start_ros2.sh Sure, I have changed the IP address to localhost and also completed his instructions. However I have forgotten to add the point_cloud_interface parameter to be launched. Now I fixed this, lack of attention...

3. In the videos, the launch file and the sensor params also got modified. i. autoware.universe.openplanner/src/launcher/autoware_launch/autoware_launch/launch/autoware.launch.xml ii. autoware.universe.openplanner/src/sensor_kit/sample_sensor_kit_launch/sample_sensor_kit_launch/launch/gnss.launch.xml iii. autoware.universe.openplanner/src/sensor_kit/sample_sensor_kit_launch/sample_sensor_kit_description/config/sensor_kit_calibration.yaml iv. autoware.universe.openplanner/src/sensor_kit/sample_sensor_kit_launch/sample_sensor_kit_description/config/sensors_calibration.yaml

Oh that is new for me. I have just seen in the video he modifying the autoware-carla.launch.xml to be similar to autoware.launch.xml and the file start_ros2.sh. In addition I have also done the map changes he did (renamed and put inside Data folder). I have also dowloaded his additional autoware.contents and darknet folders.

The issue now is withing Python version and egg files. If I export and use eggfile 3.7, I have the error below:

If I export the 2.7 egg file and run with python2.7, the scenario runner module is not found, and the same history starts again:

About the ROS version, I was using the ROS2 Galactic to maintain the same on of Mr. Darweesh. However I have read the Leaderboard documentation and it was said it just worked with ROS2 foxy. But for me this is not the problem, I can reinstall galactic and source it, if the case...

about python2 vs python3 I am afraid that I will need to reinstall requirements.txt from each package (op_agent, op_bridge, leaderboard, carla) et using pip2 install -r \folder

[yuting-fu]

Hi @marcusvinicius178 , I didn’t have time to check the details of your question, but I think your simulation issue could be similar to the problem discussed here: https://github.com/hatem-darweesh/op_bridge/issues/7

You may need to comment out some code to work around it.

[marcusvinicius178]

Thanks @yuting-fu I will take a look! I have in this moment switched to python2. However may changing from 3.8 to 3.7 as the provided document says, do the trick...let's see

[yuting-fu]

And ROS2 Galactic or Foxy both works for sure in this case. Also, CARLA 0.9.10.1 and 0.9.13 both work with Hatem’s implementation. That’s not an issue. But you do need to make sure your .egg version matches with your own setup.

[marcusvinicius178]

I have just updated my comment and will repeat here @yuting-fu about python2 vs python3 I am afraid that I will need to reinstall requirements.txt from each package (op_agent, op_bridge, leaderboard, carla) et using pip2 install -r \folder Just afraid to crash everything

Do you think is it the case?

[yuting-fu]

Sorry I don’t remember if python2 worked in this setup. I’m using python 3.7. I’d recommend to go for python3. You may encounter more problems if you use python2.7, it’s a bit outdated.

[yuting-fu]

Again, make sure to clean up your environment. I guess your environment is a bit messed up again. The python3 error you got showed that the script still wanted to load 2.7 modules.

[marcusvinicius178]

Yes this is the issue it is loading the 2.7 modules. I don't know how to force it runs from 3.7. I have already exported the 3.7 egg files.

I have issued on terminal:

export PYTHONPATH=$PYTHONPATH:${CARLA_ROOT}/PythonAPI/carla/dist/carla-0.9.13-py3.7-linux-x86_64.egg

And I have even deleted the 2.7 linux egg files from .cache folder and from simulator folder. However for some reason is trying to get just it now, and not the right one = 3.7

[yuting-fu]

You can try opening a new terminal and run everything again over there.

[yuting-fu]

You can do $ echo $PYTHONPATH in the terminal and see what you’ve got. Using a new terminal is the easiest way to have a clean slate and to start all over again.

[marcusvinicius178]

@yuting-fu almost there

Now I got permission denied error. I run with sudo it does not locate the file

This was the error when I closed the terminal

I will try to install python 3.7 as the guy did. Maybe fix it...

[yuting-fu]

I guess your pythonpath is set in the user space, not in the root. I don’t remember running autoware as root. Try to run autoware script without sudo

[yuting-fu]

Or did you build autoware with sudo? You should be able to do colcon build inside the autoware container without sudo.

[marcusvinicius178]

no, I built without sudo. I don't understand, now I have installed python3.7. Any of Python versions: 2.7, 3.7 or 3.8 are being able to recognize the modules properly. The same history again....

Python 2.7 don't see "util" module Python 3.7 don't see "srunner" module Python 3.8 don't see the leaderboard module

I have exported the scenario runner and leaderboard module within the same path to that one I was reaching the same error message you got! Then your trick to comment the lines would fix my issue. But Now I am 2 steps back again

[marcusvinicius178]

Hey @xpharry @yuting-fu I finally was able (I guess) to launch Autoware + CARLA

However anything happens (rviz did not open, etc)

What must I do to see ROS acting? That is working?

In my case the map, etc was not loaded....

[xpharry]

https://github.com/carla-simulator/leaderboard/issues/142#issuecomment-1319176482

It really depends on your ROS version. Because in Ubuntu 20.04 and ROS2 (galactic or foxy) python version is 3. So I recommended you switch to 3 to be consistent.

[xpharry]

Hey @xpharry @yuting-fu I finally was able (I guess) to launch Autoware + CARLA

However anything happens (rviz did not open, etc)

What must I do to see ROS acting? That is working?

In my case the map, etc was not loaded....

You should be able to see Autoware logo show up and then Rviz.

[marcusvinicius178]

Yes @xpharry I have updated the picture it was missing specify the launch file...Now I have specified "autoware.launch.xml" however there is still something wrong with the arguments...I will try to see what is.

I don't know bu something with the map argument?

How did you set guys? @xpharry @yuting-fu

[marcusvinicius178]

oh my code I forgot the most important! The launch word. I am sorry!

[yuting-fu]

You need to pass the map name and path as arguments when running the launch file. Also the vehicle model. There should be an example shown in Hatem’s video tutorials. Look carefully when he’s showing the commands. It’s a bit trivial, I don’t have the commands at hand at the moment.

[marcusvinicius178]

Ok now I have runned with the correct commands. But now I think there is really an issue with python 3.8

[marcusvinicius178]

@xpharry @yuting-fu I had accidentally delete /usr/bin/python folder....in other words removed python3.8 which was linked. The result is that I have broken my system when installed 3.7 and set as "default". The 3.8 version was the base of many Gnome applications....so trying to install anything I have a broken system....

But no problem tomorrow I am going to try a new approach. I have already uploaded my Autoware repo to drive. Then tomorrow I will reinstall Ubuntu, just pull compile the packages and download the simulator again.

My question comes now. You said you have used python3.7 to reproduce the results.

I have also downloaded python3.7 using "sudo apt-get". However I noticed that using this way does not Download python3.7 config folders then it is not effective symlink this version as default python. I have later downloaded python from internet repo and installed completely (python 3.9 and 3.7) this other method downloads everything and is possible to let is as default. However I am afraid to do this and brake the system again.

I am thinking in the following approach: crate a virtual environment on top of my repository (root/parent folder which is "hatem-repos", then enter into this folder activate the python 3.7 virtual environment and install the required modules to run the application (Autoware Universe).

Considering my simulator Carla 0.9.13 is under /opt/carla-simulator/ folder. I ask must i create a python 3.7 virtual environment also on top of "carla-simulator" folder?

Will this approach be enough? Or are there other methods (better)

I would be grateful if you could summarize steps that I could try.....I mean I am going to start by point 0, so If I follow the correct steps now, the integration should work, right?

Thanks very much again for the support.

[xpharry]

I was only to suggest you to switch to Python 3, because ROS2 foxy on Ubuntu 20.04 is using Python 3. Wasn't expecting you to make so mach effort on different Python 3.x.

[marcusvinicius178]

Hi @xpharry I am still facing issue. I have created a python3.7 virtual environment over hatem-repos folder. However I am facing some issue with rclp

It seems that ros Galactic is using a rclpy built over a Python3.8 version. However my venv is over 3.7 and I am also running the script using python3.7 or python3. The same result...

It seems this solved the issue: https://answers.ros.org/question/355064/ros2-modulenotfounderror-no-module-named-rclpy_rclpy/

[marcusvinicius178]

Hi @xpharry, @yuting-fu , @hatem-darweesh it seems the last comment fixed the issue: It is not possible run the script in a virtual environment of python 3.7, because rclpy is built with the Python3.8 version. Then I had to deactivate the virtual environment and run the script with Python3 In addition, it was necessary to install required python modules it was missing on documentation:

pip3 install py-trees
pip3 install tabulate
pip install transforms3d
sudo apt-get install ros-galactic-sensor-msgs-py

However there is an Agent Error as you can check and Autoware-Rviz does not open...Why?

The Error zoomed

It seems a TF error Some suggestion why map and base_link are not connected?

I have already modified the Data/ maps folder with the proper lanelet2 names, etc..... I don't understand what is going on. I have just not understood why mr. Darweesh asks to dowload the darknet folder. Because I have not seen in tutorial where he used it? I have just dowloaded and put inside "hatem-repos" folder...

Have I missed some step? Thanks in advance.

[yuting-fu]

The TF errors are there because the map is not loaded yet and no TFs are published yet. No need to worry about those errors. As to the agent error, did you comment out the lines as mentioned here? https://github.com/hatem-darweesh/op_bridge/issues/7#issue-1403919371

[xpharry]

Step 1: Build autoware.universe.openplanner

Please use "open_planner" branch and follow the instruction in https://autowarefoundation.github.io/autoware-documentation/main/installation/autoware/source-installation/

The content in file of autoware.universe.openplanner/src/launcher/autoware_launch/autoware_launch/launch/autoware.launch.xml should be replaced with that in autoware.universe.openplanner/src/universe/external/open_planner/op_launch/autoware-carla.launch.xml

In file of autoware.universe.openplanner/src/sensor_kit/sample_sensor_kit_launch/sample_sensor_kit_launch/launch/gnss.launch.xml

Modifiy

  <arg name="coordinate_system" default="1" description="0:UTM, 1:MGRS, 2:PLANE"/>

as：

  <arg name="coordinate_system" default="2" description="0:UTM, 1:MGRS, 2:PLANE"/>
  <arg name="plane_zone" default="0"/>

Step 2: Clone op_bridge and op_agent

$ git clone https://github.com/hatem-darweesh/op_bridge.git -b ros2
$ git clone https://github.com/hatem-darweesh/op_agent.git -b ros2

1. Download map data and pretrained weights (from https://drive.google.com/drive/u/0/folders/1Or0CMS08AW8XvJtzzR8TfhqdY9MMUBpS). Put them inside op_agent and merge with original folders with the same name. The raw map data is from: https://github.com/carla-simulator/carla-autoware for your reference.

2. Change the map name as the tutorial video shows.

3. The following two scrips need some modification based on your local setup.

   a. op_bridge/op_scripts/run_exploration_mode_ros2.sh Change the simulator local host ip.

   b. op_agent/start_ros2.sh Change the autoware.universe.openplanner source path. Change the map path in ros2 launch command.

Step 3: Modify sensor's setup.

Dr. Hatem has answered hrere: https://github.com/ZATiTech/open_planner/issues/2#issuecomment-1256142611

[xpharry]

I suggest you re-watch the video to make sure you would not miss any detail.

[marcusvinicius178]

The TF errors are there because the map is not loaded yet and no TFs are published yet. No need to worry about those errors. As to the agent error, did you comment out the lines as mentioned here? #7 (comment)

Yes I have commented these lines on op_agent script. Then one of the errors disappeared. Afterwards I runners. However the TF error with map and base link persisted.

[marcusvinicius178]

Step 1: Build autoware.universe.openplanner

Please use "open_planner" branch and follow the instruction in https://autowarefoundation.github.io/autoware-documentation/main/installation/autoware/source-installation/

The content in file of autoware.universe.openplanner/src/launcher/autoware_launch/autoware_launch/launch/autoware.launch.xml should be replaced with that in autoware.universe.openplanner/src/universe/external/open_planner/op_launch/autoware-carla.launch.xml I have done this ✅ In file of autoware.universe.openplanner/src/sensor_kit/sample_sensor_kit_launch/sample_sensor_kit_launch/launch/gnss.launch.xml

Modifiy

  <arg name="coordinate_system" default="1" description="0:UTM, 1:MGRS, 2:PLANE"/>

as：

  <arg name="coordinate_system" default="2" description="0:UTM, 1:MGRS, 2:PLANE"/>
  <arg name="plane_zone" default="0"/>

I have also done this ✅

Step 2: Clone op_bridge and op_agent

$ git clone https://github.com/hatem-darweesh/op_bridge.git -b ros2
$ git clone https://github.com/hatem-darweesh/op_agent.git -b ros2

I have done this. However instead use this command I navigated manually tô ros2 branch and downloaded and extracted the zip file into the proper folder = "hatem-repos/open_planner" ✅

1. Download map data and pretrained weights (from https://drive.google.com/drive/u/0/folders/1Or0CMS08AW8XvJtzzR8TfhqdY9MMUBpS). Put them inside op_agent and merge with original folders with the same name. The raw map data is from: https://github.com/carla-simulator/carla-autoware for your reference.

2. Change the map name as the tutorial video shows.

OK I have downloaded these folders and modified them as the tutorial video displayed. However I have not put the "autoware-content" and "darker" folders inside op_agent but inside "hatem-repos" folder. Just the maps I have put inside Data folder as the video tutorial showed ✅.

3. The following two scrips need some modification based on your local setup. a. op_bridge/op_scripts/run_exploration_mode_ros2.sh Change the simulator local host ip. b. op_agent/start_ros2.sh Change the autoware.universe.openplanner source path. Change the map path in ros2 launch command. Ok I have done all these previous steps....changed the path to roslaunch file, set IP to localhost,etc...✅

   Step 3: Modify sensor's setup.

   Dr. Hatem has answered hrere: ZATiTech/open_planner#2 (comment)

I believe the issues are here. Because in video tutorial Mr. Hatem has not showed the modifications he has done in sensors yaml file. At least I have watched the videos numerous times and have not seen. I am going to try in next hours and see if these paremeter modifications fix this TF map-base_link issue.

Thanks very much for the provided check list. I hope to solve this soon!

[marcusvinicius178]

Step 1: Build autoware.universe.openplanner

Please use "open_planner" branch and follow the instruction in https://autowarefoundation.github.io/autoware-documentation/main/installation/autoware/source-installation/ The content in file of autoware.universe.openplanner/src/launcher/autoware_launch/autoware_launch/launch/autoware.launch.xml should be replaced with that in autoware.universe.openplanner/src/universe/external/open_planner/op_launch/autoware-carla.launch.xml I have done this ✅ In file of autoware.universe.openplanner/src/sensor_kit/sample_sensor_kit_launch/sample_sensor_kit_launch/launch/gnss.launch.xml Modifiy

  <arg name="coordinate_system" default="1" description="0:UTM, 1:MGRS, 2:PLANE"/>

as：

  <arg name="coordinate_system" default="2" description="0:UTM, 1:MGRS, 2:PLANE"/>
  <arg name="plane_zone" default="0"/>

I have also done this ✅

Step 2: Clone op_bridge and op_agent

$ git clone https://github.com/hatem-darweesh/op_bridge.git -b ros2
$ git clone https://github.com/hatem-darweesh/op_agent.git -b ros2

I have done this. However instead use this command I navigated manually tô ros2 branch and downloaded and extracted the zip file into the proper folder = "hatem-repos/open_planner" ✅

1. Download map data and pretrained weights (from https://drive.google.com/drive/u/0/folders/1Or0CMS08AW8XvJtzzR8TfhqdY9MMUBpS). Put them inside op_agent and merge with original folders with the same name. The raw map data is from: https://github.com/carla-simulator/carla-autoware for your reference.

2. Change the map name as the tutorial video shows.

OK I have downloaded these folders and modified them as the tutorial video displayed. However I have not put the "autoware-content" and "darkenet" folders inside op_agent but inside "hatem-repos" folder. Just the maps I have put inside "hatem-repos-Data-subfolder..." folder as the video tutorial showed ✅.

3. The following two scrips need some modification based on your local setup. a. op_bridge/op_scripts/run_exploration_mode_ros2.sh Change the simulator local host ip. b. op_agent/start_ros2.sh Change the autoware.universe.openplanner source path. Change the map path in ros2 launch command. Ok I have done all these previous steps....changed the path on start_ros2.sh to the roslaunch file, set IP to localhost,etc...✅

Step 3: Modify sensor's setup.

Dr. Hatem has answered hrere: ZATiTech/open_planner#2 (comment)

I believe the issues are here. Because in video tutorial Mr. Hatem has not showed the modifications he has done in sensors' yaml configuration files. At least I have watched the videos numerous times and have not seen. I am going to try in next hours 🕖and see if these paremeters modifications fix this TF map-base_link issue 🔲😵

Thanks very much for the provided check list💡 I hope to solve this soon!🔓

[marcusvinicius178]

Hi guys @xpharry @yuting-fu the transform error from map (reference frame) to base_link (target_frame) persists.

To debug I have runned the command:

ros2 run tf2_ros tf2_echo map base_link

The output points that the target frame does not exist

I have also correct the parameters on sensor's configuration files (sensors_calibration.yaml and sensor_kit_calibration.yaml).

@xpharry I have also redone all the steps you sent me in previou comment. step-by step and merged the darknet and autoware-content folders... Then I tried to colcon build the packages again to test the integration and had this error.

Some idea?

[marcusvinicius178]

@xpharry @yuting-fu could you kindly tell me which

Nvidia Driver version have you used and which Cuda have you installed?

In tutorial the pre-build libraries says to install CUDA 11.6. However I had compatibility issues with Nvidia-driver-520. Which is the one that I am using. Therefore I needed to install Cuda 11.8

I guess that this Cuda and Nvidia-driver version can be the cause of my simulator-compatibility issue, pointed in previous error.

[xpharry]

Hi guys @xpharry @yuting-fu the transform error from map (reference frame) to base_link (target_frame) persists.

To debug I have runned the command:

ros2 run tf2_ros tf2_echo map base_link

The output points that the target frame does not exist

I have also correct the parameters on sensor's configuration files (sensors_calibration.yaml and sensor_kit_calibration.yaml).

@xpharry I have also redone all the steps you sent me in previou comment. step-by step and merged the darknet and autoware-content folders... Then I tried to colcon build the packages again to test the integration and had this error.

Some idea?

If all the packages are finished, then it is fine. Make sure there is no one aborted.

[marcusvinicius178]

@xpharry the simulator_compatibility pkg was aborted. I have reinstalled my OS the third time... that's why I have asked you which Nnidia-driver and Cuda version have you used ?

Have you also installed these dependencies manually?

Because from here: https://github.com/autowarefoundation/autoware/tree/main/ansible/roles/cuda#manual-installation it is said to install cuda 11.6 however it does not work with Nvidia-driver 450 or 520 which I am using right now

Then I needed to install Cuda 11.8

However Tensorrt required cuda 11.6 as written here https://github.com/autowarefoundation/autoware/tree/main/ansible/roles/tensorrt#manual-installation

and it was possible to install tensorrt with cuda 11.6 (I don't know why but the command worked, even with cuda 11.8 installed).

Then I had other issue to compile my package:

Basically tensorrt and simulator_compatibility packages are not being built because of driver of nvidia and cuda versions

Please if you have time take a look at my bash installation file, to catch what is wrong. Which I am producing to make life easier of next users...(It is not totally complete)

https://drive.google.com/file/d/1kJcUKuS-bnl0hfXt6uYeGBbcRiQbOobC/view?usp=share_link

[xpharry]

Can you show your "map_data" folder? And how is the file of op_agent/start_ros2.sh modified?

[xpharry]

simulator_compatibility

As I see in your screenshot, simulator_compatibility was finished and had some stderr, which is fine. I was asking if any was aborted.

[marcusvinicius178]

simulator_compatibility

As I see in your screenshot, simulator_compatibility was finished and had some stderr, which is fine. I was asking if any was aborted.

Now tensorrt_yolo is not compiling with nvidia driver 520 and cuda 11.8 but I am not sure if this is the reason

About the files I have modified all them according to tutorial you can check them here:

https://drive.google.com/file/d/1CquqRtPbBJEP4S6UUqj2Lh1ajwOvieoA/view?usp=share_link

Specifically the one you asked me:

[marcusvinicius178]

Hi @xpharry

What is really crazy is that I have used the commands:

    sudo update-alternatives --install /usr/bin/python python /usr/local/bin/python3.7 1
    sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.8 2
        sudo update-alternatives --config python 

And selected python 3.7 to be the default in my system. I hoped with this fix all the import modules issue I am experiencing, even doing in a virtual environment or running the script with Python3 as you suggested.

For my surprise the tensorrt yolo package was not compiled as I said previously. But now I have switched again to option2 = python3.8 and as a magic the tensorrt and all other pkgs were compiled

In this moment I am going to run all the scripts again and check if I get some positive result.