lgsvl / simulator

A ROS/ROS2 Multi-robot Simulator for Autonomous Vehicles
Other
2.28k stars 781 forks source link

AGENT not recognized by ROS #1401

Closed marcusvinicius178 closed 3 years ago

marcusvinicius178 commented 3 years ago

Hi experts. I am using Python API scripts with AutowareAI 1.14 version in integration with the SVL simulator.

It's almost everyhting working correct. With 2 exceptions I had lately:

1 - Regarding Sensors data: It seems that ROS is not having access to the sensor data from SVL simulator, or the SVL simulator perception sensors are not tracking the pedestrian agent:

https://get-help.robotigniteacademy.com/uploads/default/original/2X/c/c183bd36e0fd60b08ec555672ded243f567968cc.jpeg https://youtu.be/o_sXoUI8HG4

In this short video (at the end) you can check that the pedestrian is not recognized in ROS simulator (rviz), just the previous dowloaded map (that is uploaded locally previously through the runtime_interface). Can be maybe missing some python script code to enable sensors communication ( I have set the line: sensors = ego.get_sensors() ) to get sensor data , is this enough? Or is this more probably an error of subscribe to the correct laser topic (correlated to vehicle-sensor.json topics) in rviz?

In order to debug, I tried to add a bigger vehicle:

collision

But as you can check the collision was not avoided. Then I have taken a look into plugins tab in SVL simulator tab, and found the link to get more info about lidar: https://www.svlsimulator.com/docs/simulation-content/sensors-list/#lidar

I checked that the data are published to /point_cloud topic Then I added the point_cloud_1 plugin in rviz and subscribed to this topic. In a hope to see if the map gets updated to spawn in rviz the NPC agent. If you take a look in rviz when the car approachs the NPC, this vehicle is tracked by rviz (its shape is in a dotted pink square). *However I still have the warning in PERCEPTION module, specifically the Occupancy Grid Map : Message: No map received.

Should I subscribe to another topic in occupancy grid map instead the default /semantics/costmap...? Would you have please some hint to fix this detection issue? Or an example of detection already working that you could provide me?

It seems from my perspective that is more related to Autoware ready to use packages with lidar than an SVL issue. Other users also had this problem. And I would solve with something similar to this:

https://answers.ros.org/question/367056/costmap_generator-workbut-it-topic-semanticscostmap_generatoroccupancy_grid-do-not-have-data/?answer=377423#post-id-377423

Summarize:On my_sensing.launch add a subscriber to Simulator/Velodyne/Point_Cloud topic, later convert this data to a /LaserScan topic in a way that the nav_msgs/OccupancyGrid topic is able to receive this lidar data and update the map to make ROS/RVIZ recognize the Agents spawned into the simulation.

However I believe that is something already done to use (regarding lidar). Could you please assist me to find out what pkgs/scripts to use? Or code to use in sensing.launch file?

Thanks very much in advance

marcusvinicius178 commented 3 years ago

Hi experts. I am using Python API scripts with AutowareAI 1.14 version in integration with the SVL simulator.

It's almost everyhting working correct. With 2 exceptions I had lately:

1 - Regarding Sensors data: It seems that ROS is not having access to the sensor data from SVL simulator, or the SVL simulator perception sensors are not tracking the pedestrian agent:

https://get-help.robotigniteacademy.com/uploads/default/original/2X/c/c183bd36e0fd60b08ec555672ded243f567968cc.jpeg https://youtu.be/o_sXoUI8HG4

In this short video (at the end) you can check that the pedestrian is not recognized in ROS simulator (rviz), just the previous dowloaded map (that is uploaded locally previously through the runtime_interface). Can be maybe missing some python script code to enable sensors communication ( I have set the line: sensors = ego.get_sensors() ) to get sensor data , is this enough? Or is this more probably an error of subscribe to the correct laser topic (correlated to vehicle-sensor.json topics) in rviz?

In order to debug, I tried to add a bigger vehicle:

collision

But as you can check the collision was not avoided. Then I have taken a look into plugins tab in SVL simulator tab, and found the link to get more info about lidar: https://www.svlsimulator.com/docs/simulation-content/sensors-list/#lidar

I checked that the data are published to /point_cloud topic Then I added the point_cloud_1 plugin in rviz and subscribed to this topic. In a hope to see if the map gets updated to spawn in rviz the NPC agent. If you take a look in rviz when the car approachs the NPC, this vehicle is tracked by rviz (its shape is in a dotted pink square). *However I still have the warning in PERCEPTION module, specifically the Occupancy Grid Map : Message: No map received.

Should I subscribe to another topic in occupancy grid map instead the default /semantics/costmap...? Would you have please some hint to fix this detection issue? Or an example of detection already working that you could provide me?

It seems from my perspective that is more related to Autoware ready to use packages with lidar than an SVL issue. Other users also had this problem. And I would solve with something similar to this:

https://answers.ros.org/question/367056/costmap_generator-workbut-it-topic-semanticscostmap_generatoroccupancy_grid-do-not-have-data/?answer=377423#post-id-377423

Summarize:On my_sensing.launch add a subscriber to Simulator/Velodyne/Point_Cloud topic, later convert this data to a /LaserScan topic in a way that the nav_msgs/OccupancyGrid topic is able to receive this lidar data and update the map to make ROS/RVIZ recognize the Agents spawned into the simulation.

However I believe that is something already done to use (regarding lidar). Could you please assist me to find out what pkgs/scripts to use? Or code to use in sensing.launch file?

I have found these links that are very closed related to set the occupancy grid map:

https://gitlab.com/autowarefoundation/autoware.ai/core_planning/-/tree/7718e726716f77d4ebaef2a7649d55157327a9da/costmap_generator

https://autoware.readthedocs.io/en/feature-documentation_rtd/DevelopersGuide/PackagesAPI/semantics/road_occupancy_processor.html

Then I tried to create the OccupancyGrid map and had partial success. Besides the car keeps colliding, once the OccupancyGrid map did not recognize the AGENT (car) as an obstacle (It assigned the value 0 = not occupied) to the Agent. I have followed the tutorial of these links step-by-step and launched the launch files which enabled the nodes baive_motion_predict and ray_ground_filter to send data to the topics:

1 - /points_no_ground 2 - /prediction/moving_predictor/objects Also enabled the optional topic: 3 - /tf And feeded this topic with a forced transform from base_link to velodyne (Once I often received errors of this TF in terminal: Lookup would require extrapolation into the past. Requested time 1620083704.499075072 but the earliest data is at time 1620083727.652629207, when looking up transform from frame [velodyne] to frame [base_link] ):

In hope to fix this I have issued:

rosrun tf static_transform_publisher 0.0 0.0 0.0 0.0 0.0 0.0 base_link velodyne 1000 What was curious (debugging with rqt tool and rostopic echo tools) figuring out what topic was not sending data (to track why the OccupancyGrid map was not produced.) I have seen that the node ray_ground_filter (launched by runtime_manager interface) was not enough to enable the topic /points_raw generate the data. It was also necessary to launch the node ring_ground_filter in conjoint with the ray_ground_filter, afterwards the topic /points_raw started to send data. Other interesting point is that in any moment the topic /prediction/moving_predictor/objects generated data.....maybe for this reason the OccupancyGrid map usually when echoed returned any data, OR when returned just sent data (0,0,0,0,0,0....) and did not recognize the car. ???

Then I still have 2 doubts to close this issue?

1 - How to enable the OccupancyGrid properly (which modules/launch files are required to enable on runtime_manager interface)

2 - Why when the OccupancyGrid was enabled it still did not recognize the Agent

I have recorded videos of the step-by-steps and debugging procedure described above:

A) OccupancyGrid enabled randomly (Not by a logical step-by-step as tutorial said)

https://youtu.be/xbm1l_-c7I4

B) OccupancyGrid enabled but AGENT was not recognized as busy spot

https://youtu.be/ovjdi6S3wH4

Does other AD stack presents this obstacle recognition issue too?

Thanks very much in advance

marcusvinicius178 commented 3 years ago

Hi @hadiTab I have seen you were marked in this question.

I checked you are a great expert in ROS, so if you could please assist me on perception module, I will be very grateful.

I have advanced a little bit more in this issue: https://www.youtube.com/watch?v=XSKYE1Oxq-c

However I still did not figured out the reason why the /semantics/costmap_generator/occupancy_grid does not recognize the AGENT car or pedestrian as obstacle. When I echoed this topic, it returns usually an inifinite array of ZEROs values. However sometimes it returns the value 100:

`$ rostopic echo /semantics/costmap_generator/occupancy_grid

0, 0, 0, 0, 100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 100, 0, 100, 0, 0, 100, 0, 0, 0, 0, 100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 100, 0, 100, 100, 0, 0, 0, 0, 0, 100, 100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 100, 0, 0, 0, 0, 100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,....]`

The terminal that outputs the runtime_manager launched modules, always display this warning message:

Lookup would require extrapolation into the past. Requested time 1620142061.472442880 but the earliest data is at time 1620142284.496077775, when looking up transform from frame [velodyne] to frame [base_link]

And this error message:

[ERROR] [1620142288.442861565]: Failed transform from base_link to velodyne

However when I run the commands:

$rosrun tf view_frames
$ evince frames.pdf

I get this TF: frames.pdf

Afterwards I checked again if the TF exist with the cmd: $rosrun tf tf_echo base_link velodyne:

At time 1620141918.625

Also from the rqt introspect plugin, I see that the TF from base_link to localizer (Velodyne) is displayed! Therefore this should not be the reason of the OccupancyGrid map does not insert the obstacle into it:

TF_velodyne_base_link

What keep me worried and intrigues me, is the fact that:

The topic /non_ground_points returns no data when the node ray_ground_filter is triggered alone. This topic just publishes data when the topic ray_ground_filter is activated (along or without the ray_ground_filter.

In the link below the correspond features of these launch files: https://github.com/Autoware-AI/core_perception/tree/master/points_preprocessor/launch

From this Discourse: https://discourse.ros.org/t/lidar-ground-filters/8672

I ask myself if ray_ground_filter is properly working with AutowareAI, or just AutowareAuto?

Apparently I am not the first to have this issue:

https://answers.ros.org/question/367180/costmap_generator-node-do-not-work-well-2361/

https://answers.ros.org/question/367056/costmap_generator-workbut-it-topic-semanticscostmap_generatoroccupancy_grid-do-not-have-data/

How could I fix this issue? Or at least found another way to detect the Agent with AutowareAi AD Stack?

Thanks in advance

marcusvinicius178 commented 3 years ago

Hi I have answered here:

https://answers.ros.org/question/377659/occupangygrid-map-issue-ray_ground_filter-fails-to-generate-costmap-on-autowareai/

InterKnight commented 3 years ago

Hi, thank you for your answer!I tried ray_ground_filter in autoware.ai 1.14, and this is the conclusion: 1, when you enable ray_ground_filter, they may not generate /points_no_ground immediately, just wait about 5 minute with do nothing. 2, Enable ray_ground_filter before you enable other process, it can be generate /points_no_ground soon.

If you just can't get correct /points_no_ground anyway, you can try to use /ponits_raw in lidar detection and costmap_genarate. But it is not a commendable way.

marcusvinicius178 commented 3 years ago

Hi @InterKnight good to know! Thanks! I also was able to detect the pedestrian and generate the obstacle in the occupancy_grid map, as you can check in the video below after minute 25:

https://www.youtube.com/watch?v=XzzuvGpbHug&list=PL_d_YiRmUA3nnTRmF8QJKyPoEQhrdvpP2&index=7&t=1513s

Then I activated the motion planning Tab and created the trajectory to avoid the obstalce. The result is not good: The car stop behind the obstacle and does not follow the created trajectory to avoid the vehicle in front it (you can check in the same video in minute 27)

Were you able to perform an evasive maneuver using Autoware.Ai? If yes how did you achieve that?

Thanks!