Ego Vehicle not able to Engage. Do not plan the path completely Carla- Autoware Universe

[marcusvinicius178]

Hi @hatem-darweesh after struggling a lot, I was able to reproduce Carla-Autoware Universe Integration as described here https://github.com/hatem-darweesh/op_bridge/issues/8#issuecomment-1322836399

However the Vehicle was not able to complete the path planning and drive, it did not engage

Could you infer why? Maybe you had this issue in the beginning of integration? Please check my video below:

https://drive.google.com/file/d/1s8SHV-YNrNi1UfFjYXuD0I_EhtkAUEIJ/view?usp=share_link

I had the following outputs in my terminal:

[component_container-44] [pcl::KdTreeFLANN::setInputCloud] Cannot create a KDTree with an empty input cloud!
[component_container-68] [WARN] [1669147082.042754185] [control.vehicle_cmd_gate]: external_emergency_stop_heartbeat_received_time_ is false

Also

lidar_centerpoint_node-48] 2: [virtualMemoryBuffer.cpp::resizePhysical::144] Error Code 2: OutOfMemory (no further information)
[lidar_centerpoint_node-48] -------------- The current system memory allocations dump as below --------------
[lidar_centerpoint_node-48] [0x56174903eb90]:151 :ScratchObject in storeCachedObject: at ```
optimizer/gpu/cudnn/convolutionBuilder.cpp: 170 idx: 50 time: 1.17e-07
[lidar_centerpoint_node-48] [0x561724517b60]:1152 :Layer Aspects shuffle constants in internalAllocate: at runtime/common/weightsPtr.cpp: 102 idx: 0 time: 3.208e-06

And

-------------- The current device memory allocations dump as below --------------
[lidar_centerpoint_node-48] [0]:268435456 :HybridGlobWriter in reserveMemory: at optimizer/common/globWriter.cpp: 438 idx: 464 time: 0.00935888
[lidar_centerpoint_node-48] [0x302000000]:164626432 :HybridGlobWriter in reserveMemory: at optimizer/common/globWriter.cpp: 416 idx: 113 time: 0.000382956
[lidar_centerpoint_node-48] Requested amount of GPU memory (268435456 bytes) could not be allocated. There may not be enough free memory for allocation to succeed.
[lidar_centerpoint_node-48] Skipping tactic 7 due to insufficient memory on requested size of 268435456 detected for tactic 0xfbba95cf52891795.
[lidar_centerpoint_node-48] Weights [name=MatMul_0.weight] had the following issues when converted to FP16:
[lidar_centerpoint_node-48]  - Subnormal FP16 values detected. 
[lidar_centerpoint_node-48] If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
[lidar_centerpoint_node-48] 1: [convolutionRunner.cpp::executeConv::465] Error Code 1: Cudnn (CUDNN_STATUS_ALLOC_FAILED)
[lidar_centerpoint_node-48] 2: [builder.cpp::buildSerializedNetwork::636] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[lidar_centerpoint_node-48] Fail to create serialized network
[lidar_centerpoint_node-48] Fail to create context: Engine isn't created

It says Fail to create context: Engine isn't created, also there are issues with Cudnn and the message: "The current device memory allocations dump as below " means what exactly? That my NUC has not enough RAM memory?

As I explained my NUC specification is : https://www.intel.com/content/www/us/en/products/sku/202783/intel-nuc-11-enthusiast-kit-nuc11phki7c/specifications.html

Summary of Machine: Processor: intel i7 Memory: 64 GB GPU: NVIDIA RTX 2060 w/ 6GB VRAM included. memory: DDR4-3200 1.2V SO-DIMM Fequency clock: 4.70 GHz

Well other question in this regard that I would like to complement. Is about the pre-required libraries (Cuda 11.6, cuDNN, TensorRt). Considering that my terminal pointed the CUDNN_STATUS_ALLOC_FAILED I imagine that can exist some issue with compatibility of what is asked (depicted below) and what I have installed

The tutorial asks Cuda 11.6, but I have installed Cuda 11.8, because the Cuda 11.6 does not have compatibility with Nvidia-Driver 450 or 520. I had a lot of issues to find a nvidia-driver compatible with Cuda higher than 11.6. Therefore my system is Nvidia 520+Cuda 11.8

In sequence I have installed cuDNN and TensorRt. But they don't have interface with Cuda 11.8. If you try to replace 11.6 to 11.8 in this tutorial command:

cudnn_version=8.4.1.50-1+cuda11.6 tensorrt_version=8.4.2-1+cuda11.6

you get error. And I was not able to find a cudnn_version compatible with cuda11.8. Do you know which website link shows me one cudnn version compatible with cuda 11.8?

Well, I imagine you were able to run the proper Cuda 11.6 version with a nvidia-driver and the TensorRt and cuDNN libraries. For this reason I ask: Which nivida-driver have you used. And which commands did you use to install the Cuda proper version. Did you install cuda 11.6 ?

Finally I would like to understand with you guys, if the car did not engage (the module engage was not built) because:

1 - My machine is not powerful enough and should distribute carla in other machine ? 2 - My installed system is not right one (Nvidia or Cuda version or cuDNN or TernsorRt ? ) 3 - Other cause related?

Thanks in advance

[yuting-fu]

The video you made is very hard to follow. I can't really tell where and whether you clicked anywhere when doing pose estimation and goal pose.

The goal point has to be in a valid drivable space. For example, if you accidentally click on the opposite lane, it will not start planning. When setting the goal, make sure you click on the spot, and slowly drag your cursor a little bit towards the driving direction the ego car is supposed to follow. Then wait for at least 10s for the planning to complete. Please don't click around while it is planning if you are not sure the function of the buttons. You may have accidentally cancelled the planning or invalidated the plan. Sometimes the planning takes longer, could be even more than 30s. When it finishes planning, the state button will turn into light blue and show "waiting for engage". Only click on "Engage" when you see this state.

So please try again and be patient. The pose estimation is not always necessary, if by default the car is initialized in roughly the correct location already.

I don't know if at the end of your video the planning was actually finished and ready to engage. Your terminals blocked the whole view of the rviz display panel. If the planning does not go through after more than 30s, you might have some problem with the initialization of Autoware, which is a known issue mentioned in Hatem's video. The only workaround I know of, is to kill RVIZ and autoware and then relaunch using the launch script. Make sure there's no hanging process in the background before you relaunch everything. Sometimes restart (you don't have to rebuild, just quit the container and do $ docker restart CONTAINER_NAME) the Autoware container also helps.

In principle, if you see in the panel "waiting for route", the autoware initialization should be okay already, which is the case in your video. If you send a valid goal pose, it should be able to plan and eventually go into "waiting for engage" state. If this does not happen after you re-try multiple times, then you may have to dig deeper into the root cause.

[marcusvinicius178]

@yuting-fu I have done everything you said. Maybe the video don't show. I have done at least 7-10 times the same procedure. And yes I clicked in a valid place inside the road ( I have zoomed ). I have also reboot many times to avoid let some old related process running.

I believe the issue is not about click on engage several times, because in my case different from Harem's video tutorial I waited for more than 1 minute and the blue light has never been triggered.

I believe the issue is related to the output in terminal: "Engine isn't created" and cudNN. That's why I have asked you which Nvidia-driver, Cuda, cudNN and TensorRt versions have you installed ???

Because in my point of view this can be the root cause of my issue. And I would like to compare your system with mine.

Thanks for the reply

[yuting-fu]

My environment is probably much older than yours. It's done a few months ago. My Nvidia driver: +-----------------------------------------------------------------------------+ | NVIDIA-SMI 510.85.02 Driver Version: 510.85.02 CUDA Version: 11.6 | |-------------------------------+----------------------+----------------------+ If you must know, the autoware image I used back then was: ghcr.io/autowarefoundation/autoware-universe:galactic-20220816-cuda@sha256:31db011573652a33339d546db065aba2e2604ddd7009f2f2036fee3c53864d8f

I didn't change anything in the autoware amd64.env, just used the default:

rosdistro=galactic
rmw_implementation=rmw_cyclonedds_cpp
base_image=ubuntu:20.04
cuda_base_image=nvidia/cuda:11.6.2-devel-ubuntu20.04
cuda_version=11.6
cudnn_version=8.4.1.50-1+cuda11.6
tensorrt_version=8.4.2-1+cuda11.6

[marcusvinicius178]

Hi Mr @yuting-fu thanks a lot to share your system properties. My intuition tells me that the Cuda 11.8 and Nvidia 520 installation generated compatibility issues with the cudnn and TensorRt libraries, once they were installed along cuda11.6 and not 11.8 ( I installed using the same command as yours). Tomorrow I am going to clean the Nvidia drivers and Cuda libraries and try install the driver 510 with Cuda 11.6. I hope it works ( probably won't have conflict with cudNN and TensorRt anymore). In addition I have not installed Autoware using Docker. I am installing everything from source, because my team does not trust in Docker or VM environments to embed the NUC into the vehicle.... I always prefer to work with Docker too, but I must follow orders ....

Tomorrow I will update here if the change of the system has enabled the build of Engine module and if has fixed the navigation errors...

[malayjerdi]

Hi @yuting-fu . I have the same issue. I think the nvidia driver is not issue. because I have nvidia 510. If you found some solution please tel me about that.

[yuting-fu]

@farhang760 maybe check your initial pose as well, if you are using Town01 map and running the CARLA leaderboard agent from Hatem, your initial pose seems a bit off. Although I'm not sure if that is the root cause of your issue. You can try publishing the initial pose estimation from the command line to adjust it. I don't run the whole setup right now, the data and screenshot are from my old notes. See the small circle in the picture, on the right lane, that is the initial spot. The position message is roughly:

$ ros2 topic echo /initialpose2d
header:
  stamp:
    sec: 0
    nanosec: 0
  frame_id: map
pose:
  pose:
    position:
      x: 339.72259521484375
      y: -229.94802856445312
      z: 0.0
    orientation:
      x: 0.0
      y: 0.0
      z: 0.6831836460376832
      w: 0.7302466061452512
  covariance:
  - 0.25
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.25
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.0
  - 0.06853891909122467

[malayjerdi]

Thnaks @yuting-fu for the comment. I am using Town 01 and when I publish your position this happen:

[yuting-fu]

Hmmm, weird... And if you set a goal pose, the planning still doesn't go through? I'm using the op_agent for CARLA leaderboard, which Hatem integrated with autoware, the script run_route_scenarios_ros2.sh. You may need to modify some sensor position configurations in op_ros2_agent.py and the imu_msg ROS convention for CARLA, if you see some errors regarding the imu. The CARLA leaderboard scenario runner will initiate the starting point, and I think the one I sent you is roughly the initial pose. You can also check that by navigating in CARLA simulator and see how the surrounding looks like

[malayjerdi]

I am using the run_exploration_mode_ros2.sh from hatem package. could you please send me your op_bridge package?

[malayjerdi]

@yuting-fu ,Maybe is because of this. I mean it show an obstacle on the road

[marcusvinicius178]

Hi @yuting-fu , @farhang760 you were right I have unistalled nvidia driver 520 and cuda 11.8 and reinstalled nvidia-driver 510 with cuda 11.6. However the issue persists.

I have recorded 2 videos more slowly, where I have echoed the localization and it is ok! The Ego vehicle returns the pose:

But when I have echoed planning topics, most of these topics returns: "The message Type is invalid

The goal pose is never catched by the vehicle, otherwise I believed it should be displayed when echoed, i.e, goal_pose_y = 80 and goal_pose_x = 30, orientation_x = 90 .....etc..... (Just an example), but it gets nothing.

Do you know the command to send the goal pose in ros2 by hand, I mean issue it in terminal, instead of using the goal pose icon in Rviz? Maybe it work

Another point, that maybe the root cause of this issue is the progressive construction of the path planning. I believe that when the simulator and the Autoware Stack are runned in the same machine (even a powerful machine as my NUC), the path planner ran out of memory. My conclusion is because I have already done simulations using Autoware.Ai, Autoware.Auto and Apollo.Auto integrated with LGSVL simulator during my master thesis, years ago...And when I have rent one of the most paperspace's powerful machines in the cloud, the planners worked, but they didn't work in a single machine!

In addition, they worked better and more smoothly, when I installed the simulator in a desktop and the Autoware stack on my Laptop. I guess the planning is not being successfully built because of this output message in terminal:

Cannot create a KDTree with an empty cloud

Is this KDTree the tree of the planner? It seems that the planner's waypoints are empty, were not built.

My debug videos are here:

https://drive.google.com/drive/folders/1RS1QtAiy5gREVknNLYoO1L-gfnTMS6cR?usp=share_link

In the most large video I have runned the stack and simulator for first time. The second one (shorter with the camera image working) , it was when I closed and relaunched (just done again because Mr.Hatem said in tutorial that sometimes is needed relaunch to make it work).

Finally I would like to know with you @farhang760 if are you running in 2 machines or just one machine as me? Maybe is this the issue @yuting-fu ? Or what do you think?

And just to complement @farhang760 , I guess the obstacle or the occlusion spot are not the problem, considering I have sent another goal point very close to the Ego Vehicle (without obstacles ahead it) and it has not driven to it in the same way (my second video).

[marcusvinicius178]

I was also running run_exploration_mode_ros2.sh but now I have runned the script "run_route_scenarios_ros2.sh" and I had the same result.

But now, I had the following error in terminal:

Message Filter dropping message: frame 'base_link' at time 5.900 for reason 'the timestamp on the message is earlier than all the data in the transform cache'

This attempt was recorded here: https://drive.google.com/file/d/1kT7hXNPi5WbavFL9C9p32Sq8nB18Q26D/view?usp=share_link

After kill the process, I had this "Results" ouput in terminal:

It seems again some RAM memory issue, right?

[malayjerdi]

Hi @marcusvinicius178 , Thanks for your comments. I am running the carla and autoware in the same machine. and the goal topic is : ros2 topic echo /planning/mission_planning/goal

[marcusvinicius178]

Hi @marcusvinicius178 , Thanks for your comments. I am running the carla and autoware in the same machine. and the goal topic is : ros2 topic echo /planning/mission_planning/goal

Yes did you get some info from this topic? I got empty messages. Were you able to run one of the op_scripts successfully? If yes please tell me. I will try to run in 2 machines and see if I get a different result (I know Mr.Hatem has ran in different machines as he said in his tutorial video).

[yuting-fu]

I run CARLA and Autoware in the same machine, autoware in a docker container, carla on the host or in a seperate container, both work, doesn't matter. It also works if I run them on two different machines. To learn how to properly publish to a ROS2 topic, you can spend some tome following this tutorial. Also make sure you have the right msg package installed and sourced in your terminal environment. To check which topic you need to debug/publish/subscribe, you can run rqt and see the node graph and topic graph. There are many tutorials online. It's just the same as in ROS1, if you have used those tools before.

[malayjerdi]

Unfortunately doesnt work

[marcusvinicius178]

I am not running with Docker, but from source. Maybe in Docker this planning message is correct. I believe I will need to spend some time debugging the controller/input and the planning/output messages.

This invalid message type can be the error. However I have installed all the pre-requisites manually. I have no idea where should I "dowload the proper planning messages package". Maybe they come with ROS2 Humble instead of Galactic? @yuting-fu which is the hint to debug this msg type? I have already used RQT and ROS1 tools, but this case of invalid msg type is new for me. Also because I imagined that all the messages were already built with the tutorial pkgs.

@farhang760 did you try Docker or also from Source?

[malayjerdi]

No, I am using Autoware from the source

[yuting-fu]

Did you check if you have that message type installed already in your environment? If you installed the Autoware version for Galactic, in principle you shouldn't have issue with message types. But you can try to build Autoware with some later or earlier commits. Check in Autoware repository their commit messages and try to use a commit that looks more reliable. Using docker or not is not the issue.

The goal message type is not the root cause at all. Both of you may have issues with the sensor configuration, such as lidar and imu. I don't recall run into any issue with KDTree, I have no idea what it is. With RQT you may be able to figure out which module triggers the error.

If you have tried Apollo-Carla integration, you probably know the coordinate system transformation is tricky. I modified some code in op_ros2_agent.py to correct the coordination system conversion, removing a minus here and there, and modified some yaw angles for the lidar and IMU.

It's been more than three months now I can't recall more details. Maybe someone else can provide more up to date information.

[marcusvinicius178]

Hi Mr. 2 @yuting-fu have you remained these files in your computer? op_ros_agent.py, Lidar and IMU config files? If yes could you please provide me? My email is marcusvini178@usp.br

In addition I have runned this launch file https://github.com/autowarefoundation/autoware.universe/blob/main/simulator/simple_planning_simulator/launch/simple_planning_simulator.launch.py in my machine.

However I had errors. Do you know if this internal simulator is not working for Autoware.Universe? I just need store in a rosbag the control messages to build a hardware (controller). Do you know if these control messages remained the same for Autoware Projects (Autoware.AI, Autoware, Autoware.Auto and Autoware.Universe)? Or do you have a rosbag of control messages from Universe to provide ?

[yuting-fu]

Try rolling back a few modifications Hatem did in his latest commit. Basically change your code back to the code highlighted in red (not the full line of code, only the changes shown in the screenshot) in op_ros2_agent.py: Regarding the Autoware planner example, I was able to launch it, but I don't rememberl if I had errors anymore. Anyway It was not very useful in my use case so I quickly stopped using it. I don't know if the control messages are different or not. ROS1 and ROS2 definitely has different types. For details, you need to check the topic names and message types in all the projects you mentioned.

[marcusvinicius178]

Mr @yuting-fu I have at first done just the modifications you showed in your screenshot (from line 307 to 419). I had a similar result to Hatem's video regarding the vehicle orientation . However the vehicle still don't navigate.

I have also tried to get all the modifications (in addition of your screenshot):

I have fetched the last commit of Mr.Hatem called "Fix IMU data Orientation", which have the modifications in the 3 scripts your link refers: op_ros2_agent.py, op_bridge_ros2.py and run_exploration_mode_ros2.py

Then I have copied these 3 files to keep them safe, Then I have returned to my original commit (with the video tutorial codes) Finally I have replaced these 3 files to the local repository folders (to my commit).

The result is that Autoware and Rviz worked worse and did not tlocalize the vehicle properly. My question is if the modification is just on the parameters showed in your screenshot or should I modify other files?

[yuting-fu]

No, I only suggested to modify the parts I showed in the op_ros2_agent.py file. If it doesn't work, I guess you had different settings in some of those sensor launch files, already mentioned in one of the issues you had before. I suggest you examine all those sensor launch files once again and make sure they are correctly configured. Also, you may need to rebuild Autoware if you change any the the launch files. Also, as I said, I ran run_route_scenarios_ros2.sh. I don't remember what I got when I ran the exploration script. They might require different configurations. I can only tell you what I had experimented with.

[yuting-fu]

Note that my autoware version was from August. Lots have changed over time. If you just git pull the latest Autoware universe version, it is a moving target. Multiple new commits are pushed to the repo every day. So you probably can't just copy anyone's (including Hatem's) step. You need to know why those changes are needed and double check if they are still needed based on the autoware code you are running right now. And, I'm not a Mr. 😃

[marcusvinicius178]

I checked that my error:

Cannot create a KDTree with an empty input cloud!

Can be related to the load of the sensor_kit_calibration.yaml file as this answer suggests:

https://answers.ros.org/question/324029/problems-encountered-with-rgbd_calibration-exception-parsing-yaml-camera-calibration-yaml-cpp-error-at-line-0-column-0-bad-conversion-error-failed-to/

I have used sublime-text editor to check which launch file or other file is calling it:

But the problem is that I have not seen any file calling this calibration file, which may is resulting in an issue with PCL and generating this KDTree issue, that you didn't have as you said previously.

Do you know which file is calling the sensor_kit_calibration.yaml. Probably it can lead to some issue with perception and then with the subsequent signal flow modules: planning and control. What do you think about?

Sorry to disturb you a lot. But I have put much effort in this simulation and use the framework would help a lot my research team.

Thanks for all your attention and replies. I am going to check later the sensor's parameters, try at least one more time....

[xpharry]

I suggest you source build the raw autoware package in a different folder and run the planning simulation in the tutorial to see if you can engage. This is to separate the issue from Dr. Hatem's packages.

[malayjerdi]

I can run the planning simulation without any error. the problem happened when I want to use Hatem package

[malayjerdi]

I found the issue. Thanks @yuting-fu and @marcusvinicius178 . I used the autoware from Hatem git . I think, it need update. I used the autoware git and checkout to awsim-stable and now it works.

[xpharry]

@farhang760 Was your issue happening in "open-planner" branch?

[malayjerdi]

@farhang760 Was your issue happening in "open-planner" branch?

yes

[marcusvinicius178]

Hi

No, I only suggested to modify the parts I showed in the op_ros2_agent.py file. If it doesn't work, I guess you had different settings in some of those sensor launch files, already mentioned in one of the issues you had before. I suggest you examine all those sensor launch files once again and make sure they are correctly configured. Also, you may need to rebuild Autoware if you change any the the launch files. Also, as I said, I ran run_route_scenarios_ros2.sh. I don't remember what I got when I ran the exploration script. They might require different configurations. I can only tell you what I had experimented with.

I have rebuilt Autoware, after do the script's changes explained in tutorial. When building the open-planner branch this time (with modified files), I had an error in lidar_centerpoint package https://autowarefoundation.github.io/autoware.universe/main/perception/lidar_centerpoint/ Then I needed to dowload and replace this pkg in autoware.universe.openplanner folder: https://github.com/autowarefoundation/autoware.universe/tree/main/perception/lidar_centerpoint

I have runned the scripts again and the engage really does not work with the open-planner branch.

@farhang760 I have switched to the autoware repository (git checkout main), fetched the main branch, and rebuilt autoware. However the vehicle still does not engage. Have you done some additional step?

@farhang760 Hmm I checked now that you have checked out into awsim-stable. I am not so good in Git. How did you find this branck and check out it? I just localized main branche.

[marcusvinicius178]

@farhang760 @xpharry I tried also to run the QuickStart Demo from AWSIM however my map was not loaded in Rviz. The issue is here: https://github.com/tier4/AWSIM/issues/38 Am I missing some argument or setting it wrong?

I guess after try this I am going to replace the autoware.universe.openplanner to the autoware folder, checkout to the aw-sim stable branch, build and finally test it...to see if it works.

[marcusvinicius178]

Hi Now with the Autoware folder built the vehicle has finally engaged, however it has not driven as can be checked in this video

https://drive.google.com/file/d/1ExAmSRvWcqKHivS7iVdUgITxLt3T5uI7/view?usp=share_link

I have modified again the scripts according to Hatem's tutorial. And I needed to include again the open_planner folder to rebuild the Autoware Universe pkgs, otherwise the carla_interface pkg won't exist and integration won't work.

My vehicle for some reason this time were not able to auto localize and I needed to use the rviz icon send estimate pose for it being able to localize. I am not sure if this is the issue, because the vehicle calculted the path and was able to engage.

The outputs from terminal:

[localization.pose_estimator.ndt_scan_matcher]: Not Converged
[ndt_scan_matcher-36] [WARN] [1669319045.972540000] [localization.pose_estimator.ndt_scan_matcher]: Nearest Voxel Transformation Likelihood is below the threshold. Score: 0.000000, Threshold: 2.300000

It seemd the ndt alorithm is not coupling the pcd map data with the current lidar point cloud data. Some suggestion?

[marcusvinicius178]

update: The AWSIM simulator worked...the vehicle engaged and drived normally as it is in my video https://drive.google.com/file/d/1vwceUrZDRwxx-4BxgPJ7SAundk9xbZvv/view?usp=share_link . The car has not driven just in CARLA until this moment for some Perception or Localization issue....????

[marcusvinicius178]

Ok my fault. I forgot to correct the Sensors configuration files inside the folder

/home/rota/hatem-repos/carla-autoware-universe/autoware/src/sensor_kit/sample_sensor_kit_launch/sample_sensor_kit_description/config

After rebuild the workspace it worked!

The video: https://drive.google.com/file/d/1C-B89ZO8Odn6oVosm_YWSoDA_kYNY1TI/view?usp=share_link

Thanks a lot guys for the assistance!!!

and Mrs. @yuting-fu you have mentioned you have already worked with Apollo-Carla sensors integration? I am very curious how you have done that because I couldn't create a real sensor bridge for a Real localization and not fake localization provided by Auro-AI bridge https://github.com/AuroAi/carla_apollo_bridge

Could you please tell me how you did that. My email is marcusvini178@usp.com or if you could reach me at linkedin: https://www.linkedin.com/in/marcus-robotics/

I would be very grateful and I would love to understand how to buidl a bridge from scratch !!!

[kshitiz-deloitte]

Hi @marcusvinicius178 , I am also facing the same issue I have made the required changes in sensor_kit. I have gone through all of the comments in this issue, but I am not able to resolve the issue (state is stuck in planning). Can you or anyone please help me resolve this issue?

[marcusvinicius178]

I would recommend you at first:

1 - Try to test Autoware along Tier 4 simulator: https://autowarefoundation.github.io/autoware-documentation/main/tutorials/scenario-simulation/planning-simulation/scenario-test-simulation/

If planning, engaging, and controlling worked with this simulator, there is still an issue with your code. After making the changes, have you rebuilt the workspace with colcon build....etc.?

If the Autoware modules didn't work with the simulators, you probably don't have the hardware requirements to run everything on just one machine.

[kshitiz-deloitte]

Thanks for reply @marcusvinicius178, I have tried rebuilding the workspace using colcon build. But still after that I am not able to engage the vehicle. I dont think the issue is with hardware. If there is any other way to resolve the issue that you know it would be helpful if you can share those. Moreover I am getting the occlusion spot message over the path. )

[marcusvinicius178]

Probably you forgot to change some code on configuration files or other modified files, such as the launch file, or the ros2_bridge.sh file or other one ( I don't remember the names)....

Or you have not checkout to the awsim-stable branch before rebuilding the pkgs. Did you?

What I can do is share my local repository, then just paste and build over yours (I mean remove your and use mine): It is available here:

https://drive.google.com/file/d/1FTq_2J3QwYWFA5MHbjnW9Kb4mEDW9t84/view?usp=share_link

You will have export issues and path issues because my user is "/home/rota". Therefore just replace rota to your username or use ($USR).

In addition you need to source the ros/distro/setup.bash and autoware/install/setup.bash everytime before running the script. Have you done this?

In my hatem-repos folder you will check I have created a bash file called correct.bash. Just source it before starting using the terminal.

The exports statements in this bash file won't work! I don't know why, but you need to copy them with right-click of mouse and paste directly in your terminal. It is the only way to export statements work.

I hope it works

[xpharry]

One fact that can help explain why previously had no issue but now have some issue such as "not able to engage" is that we are using Autoware code with the master branch which is evolving daily. That is to say, we are not using the exactly the same code if we clone it at different days. That causes some risk that the tutorial may be outdated.

That is why @farhang760 switched to a stable branch and made it work.

I have two folders in my PC. One is from maybe a week ago and one is from yesterday. The latter one give the same result that it cannot engage and the camera image doesn't show up. I will compare the two folders and find the root cause.

[kshitiz-deloitte]

Hi @marcusvinicius178, thanks for the repo, I have downloaded the repo you have provided but now I am not able to source the autoware setup. Whenever I tried to run source, I am getting the following error for every library that are inside install. {not found: "/home/ubuntu/Downloads/hatem-repos/carla-autoware-universe/autoware/install/web_controller/share/web_controller/local_setup.bash"} I checked the file (mentioned in the error above) it is there but it is empty when opened with txt editor. Can you please suggest what i am missing in this case. (I am trying to make it work and then trying to compare and understand with my repo) (PS: I have made changes in path in correct_bash.sh file.)

[marcusvinicius178]

Hi @kshitiz-deloitte, I am really sorry, but I have no idea... Probably when I built the workspace, some files in install folder were built for my system using my path...and when you are building in your system, something is being overwritten and does not work....

What I can suggest is to take my modified files and copy and paste them into your original repository I mean, check in google for compare folder/files differences. Then use the command you find over the folders you know there are differences for example the sensors_description config folder ...usually in Linux you use diff** just check https://www.networkworld.com/article/3607728/smart-ways-to-compare-files-on-linux.html

Actually to make things work you need:

1- Compare my files with your files using the diff command (of course just the files that Mr Hatem has changed in his tutorial, and the additional configuration files that he does not show in his tutorial but need to be changes (Velodyne and another one. I don't remember which but are written in some comments above...

2- Dowload autoware. Instead you download the autoware.universe.openplanner from Dr. Hatem's repo, you must download this repo:

git clone https://github.com/autowarefoundation/autoware.git
cd autoware

Then you must checkout to the STABLE branch BEFORE building the workspace with colcon....

git checkout awsim-stable

3- Now if you have done 1) the correct files modification 2) built autoware modules successfully. You just need to source the setup.bash files correctly and launch the simulator and autoware stack.

Basically, if you can reproduce this tutorial: https://tier4.github.io/AWSIM/GettingStarted/QuickStartDemo/

You will have done 90% of the work, afterward, if your Autoware-Carla does not work is just an issue of

• Have not modified all the files from tutorial correctly
• Did not build the stable branch
• Not fulfill the hardware requirements

I hope you reach this...

[kshitiz-deloitte]

Thanks @marcusvinicius178 , Now I am able to engage the vehicle. Thanks for your suggestions and time.

[marcusvinicius178]

Hi @kshitiz-deloitte I am happy you were able to engage the vehicle! If don't be ask too much, I would like to tune my skills on Linkedin profile, if you could recommend me to ROS related work: https://www.linkedin.com/in/marcus-robotics/

I have also a new page which I am still feeeding, where I get some freelance short ROS-related jobs: https://www.linkedin.com/company/mv-ros-freelancer/ I get some jobs to complement my scholarship $.... Thanks!

[cacao1987]

autoware in a docker container, carla on the host or in a seperate

@yuting-fu Hi,I also want to run autoware in a docker container, carla on the host ,but i do not know the operational procedures，can you give me an example, thanks!