Lane line detection using OpenCV

[AbBaSaMo]

• [x] Create a basic ROS node that subscribes to the Z Camera image topic
• [ ] Use OpenCV to detect lane lines from the image and output the result as a top down view of detected lanes
• [ ] convert this to an occupancy grid
• [ ] output to topic lanes/map

Dependency with https://github.com/Monash-Connected-Autonomous-Vehicle/ESDA/issues/22

[Jokua]

So far I've only managed to get the zed_ros2_wrapper package to work (this below is an rviz view of what it has). It took quite a while because it kept complaining a package was missing when I followed the tutorial, but it seems like it also required the zed-ros2-examples package. Below is a screenshot of the Rviz view and also one with my arm in it.

Rotating the camera also updates the position of the model in RVIZ and I had to rotate the view to show this result. So the IMU works pretty well too

Regarding the tutorial for setup, you can use this one but don't forget to install the zed-ros2-examples package before running the line for rviz in the tutorial. Also just replace the in the tutorial with zed2i

Also here's the rqt graph for the nodes currently running

[AbBaSaMo]

@AnthonyZhOon we can fuse the zed cams imu with the piksi imu using robot_localizations efk node. Should we? Extra source of positional data.

@Jokua with regards to the examples interface package, this is included in the ros wrapper package but as a git submodule

It's a pointer to the example interface package, but because it's just a pointer, you need to actually fetch the package i.e. pull the repo it is pointing to. This is done using the command git submodule update --init inside the zed wrapper package root

So for future reference you can copy paste the following to set up a workspace:

mkdir -p zed_ws/src; 
cd zed_ws/src;
git clone https://github.com/stereolabs/zed-ros2-wrapper.git;
cd zed-ros2-wrapper;
git submodule update --init;
cd ../../;
colcon build;
source install/setup.bash

[AnthonyZhOon]

sounds fine to fuse it since we have the transform anyways, maybe add a conditional to turn it off if it creates problems

[Jokua]

@AbBaSaMo I tried the submodule update thing, but it gives me this error for trying to get the zed_display_rviz2 runnning

This was the tutorial that I was following

Seems like the rviz display thing isn't available in that wrapper package, and another github page from these guys say that I have to install the zed_ros2_examples package anyways where that rviz package is. I wonder if i can just copy that into the zed_ros2 wrapper and it'll just work.

But eh I'll just leave it as is, I'll get to trying to get the stuff listed above working first

[Jokua]

I've managed to get a subscriber node to take the image from the zed camera's publisher node, now going to work on getting the thresholding working. This is from the zed camera's gray image nodes

[Jokua]

Just as a formal update on the progress for lane line detection, the code Jiawei wrote is generating an occupancy grid as shown below:

But it seems that the transforms from camera view to birds eye view is slightly off. Based on the region that was taken and transformed(i changed it to 0.2, 0.5 and 0.8, 0.5 for top left and top right respectively for testing), it seems to be detecting the lane lines as long as either one was in the region. However during testing i noticed that it was a bit off that I had to move more to the right in order to detect both lane lines.

Bottom line is we have to work more on the transforms, and also is it possible to have the occupancy grid have smaller pixels but still have that big grid? (sounding weird bcs idk how to word this well enough LOL)

[AbBaSaMo]

@Jokua @Jiawei-Liao

Once the occupancy grid is the right dimensions and resolution, I want this branch merged into main even if the transform to top down view is a bit off. Deadline for this is the night of Friday 26th.

Then branch out again and only one of you continue working on it until the transform issue is fixed.

[AbBaSaMo]

Now it's accruate to within 5cm given the lanes are 3-6ft wide this is like a 2.5-5% degree of error. Closing issue.