Set up & configure Autoware sensing component (archive)

[dylan-gonzalez]

Understand Autoware Sensing

See AVP Diagram (click View->Tags, hide everything but M1)

• [ ] Ensure sensor drivers are outputting correct message types on Twizy

  • [x] LiDAR as sensor_msgs/PointCloud2
  • [ ] IMU as sensor_msgs/Imu
  • [ ] GNSS as sensor_msgs/NavSatFix and autoware_sensing_msgs/GnssInsOrientationStamped
  • [ ] Cameras as sensor_msgs/Image
  • [ ] Radar as sensor_msgs/PointCloud2

• [ ] Ensure sensor drivers are outputtinf correct message types on ENV200

  • [ ] LiDAR as sensor_msgs/PointCloud2
  • [ ] IMU as sensor_msgs/Imu
  • [ ] GNSS as sensor_msgs/NavSatFix and autoware_sensing_msgs/GnssInsOrientationStamped
  • [ ] Cameras as sensor_msgs/Image
  • [ ] Radar as sensor_msgs/PointCloud2

• [ ] research the difference between using RadarTracks v Pointcloud

  • for reference our radar driver for the ENV200 is smartmicroa, which outputs a pointcloud

• [ ] Fix up launch file configurations for ENV200 & Twizy

• [ ] Configure mcav_autoware_sensor_kit

• [ ] create test cases for the CI/CD pipeline

If you need to test Autoware with a simulator, read here.

If you need any help understanding your task, please ask a lead!

[dylan-gonzalez]

Autoware is now working on the Beauty. See the wiki for instructions. Make sure to be inside the docker container first.

[AbBaSaMo]

@dylan-gonzalez any docs or people we should refer to when testing sensor drivers on the cars?

[dylan-gonzalez]

for the van: me, @hubdwoo, @Jiawei-Liao (for IMU), @AnthonyZhOon for the twizy: me, @AnthonyZhOon

Van:

• radar issue: does not power on. See the zendesk and please double check about the radar cable being connected. Login/password is here.
• cameras: currently 2 of them seem to stream at one time. Haven't properly tested them out in RVIZ yet, you just need to complete this step that I have outlined in my last comment here
• lidar: does not power on.
• IMU/GPS: works - see this issue

[AbBaSaMo]

@dylan-gonzalez what did you mean by lidar doesn't power on? it seems to be on rn and the web portal working

[dylan-gonzalez]

Oh fair dinkum

[AbBaSaMo]

Running into issues launching mcav_autonomy/autonomy_launch. Issue pinned down to streedtrone.launch.xml lines 20 and 60.

Tasks to debug include

• [x] figuring out which of the two lines is causing the issue (launching kiss_icp or streedrone vehicle model)
  • the issue is in line 60: launch imports xacro which is not listed by pip list
• [x] resolving lack of launch file/directory for kiss_icp esp if this is the issue not the issue, launch exists
• [x] resolving missing xacro dependency which is probably the issue
  • temporarily installed inside container which fixed it
  • new issues encountered regarding modules, likely in the same py launch file
  • also "substitution on args not supported" error

Once solutions found, will have to update the Dockerfiles to avoid encountering in the futur.

[AbBaSaMo]

Getting the error

@AnthonyZhOon or @dylan-gonzalez if I just want to run the various sensors should I be doing it by running the streetdrone.launc.xml or should I just launch them one by one and test their output?

Eventually the former will need to work anyway no? Seems the current issue is the fact that the roslaunch pkg does not exist for humble https://index.ros.org/p/roslaunch/#humble

[dylan-gonzalez]

Is this on the twizy?

[AbBaSaMo]

Yup on the Twizy though does that make a difference?

[dylan-gonzalez]

So from now on we don't want to use the mcav_autonomy repo the streetdrone.launch.xml file launches the sensors as well as all our other custom packages

if you want to run the lidar you can use just a velodyne driver cameras dont work at the moment not too sure actually how to run the imu

so looks like only velodyne and the CAN works atm with the twizy

[AbBaSaMo]

Running VLP16 driver on twizzy outputs the right message but there is no data. Lidar is humming so it seems to be on but not doing anything atm

[AbBaSaMo]

@dylan-gonzalez do we want to include the gryo_bias_estimator package and also update the node diagram accordingly?

[dylan-gonzalez]

If you guys think so?

[dylan-gonzalez]

11-12-23

• make sure all sensors work on the van (individually)
• make sure all sensors work on the twizy (indivudally)
• once all vehicle sensors work, update @Robin-Chea so he can make changes to the mcav_sensor_kit_launch files
• look at radar sensor fusion with/wo lidar
• @zacharyeap to comment his research

[zacharyeap]

RadarTracks

• messages convey info about tracks of detected objects through radio waves
• Contains more information about detected objects (eg. position, velocity, acceleration, size)
• Objects represented by a unique ID generated by radar
• All data of the object is relative to radar's frame of reference

PointCloud

• Represents set of datapoints in a 3d coordinate system
• Each point usually includes coordinates (x,y,z) from the sensors frame of reference
• The actual data is stored in a 1D array so the format and arrangement depends on specific fields (eg. height and width)