Implementing lane detection method for lane level localization purposes

StepTurtle commented 1 month ago

Checklist

[X] I've read the contribution guidelines.
[X] I've searched other issues and no duplicate issues were found.
[X] I've agreed with the maintainers that I can plan this task.

Description

This task focuses on the development of a ROS2 node designed to identify lanes within 2D images utilizing state-of-the-art lane detection methods.

Purpose

The primary objective is to integrate this node into the lane level localization process, specifically addressing the 2D lane detection component.

Possible approaches

Conduct manual evaluations of popular lane detection models to ascertain their suitability for our specific requirements.
Based on evaluation outcomes, proceed with the development of a rudimentary ROS2 node capable of detecting lane markings within images.

Definition of done

[X] Existing lane detection methods were looked at and decided on one of them
[ ] To use the model with ROS 2, a basic ROS 2 node was created.
[ ] Ensured that the model performs well in different light and weather conditions
[ ] Determined whether there is a need to train or fine-tune the model
[ ] Documentation is created to guide users on how to use the tool.

StepTurtle commented 1 month ago

Here are some results from popular methods which have good scores. Most of them has a high rank in Papers With Code.

This table presents several lane detection methods with various configurations. Some of the models produce segmentation results indicating whether a pixel belongs to a lane or not, while others provide arrays of lane points. I guess we can update all models to return segmentation results.

Model	Backbone	Dataset	Confidence	Video-Campus	Video-Road
CLRerNet	dla34	culane	0.4	Link	Link
CLRerNet	dla34	culane	0.1	Link	Link
CLRerNet	dla34	culane	0.01	Link	Link
CLRNet	dla34	culane	0.2	Link	Link
CLRNet	dla34	culane	0.1	Link	Link
CLRNet	dla34	culane	0.01	Link	Link
CLRNet	dla34	llamas	0.4	Link	Link
CLRNet	dla34	llamas	0.2	Link	Link
CLRNet	dla34	llamas	0.1	Link	Link
CLRNet	resnet18	llamas	0.4	Link	Link
CLRNet	resnet18	llamas	0.2	Link	Link
CLRNet	resnet18	llamas	0.1	Link	Link
CLRNet	resnet18	tusimple	0.2	Link	Link
CLRNet	resnet18	tusimple	0.1	Link	Link
CLRNet	resnet34	culane	0.1	Link	Link
CLRNet	resnet34	culane	0.05	Link	Link
CLRNet	resnet101	culane	0.2	Link	Link
CLRNet	resnet101	culane	0.1	Link	Link
Fenet v1	dla34	culane	0.2	Link	Link
Fenet v1	dla34	culane	0.1	Link	Link
Fenet v1	dla34	culane	0.05	Link	Link
Fenet v2	dla34	culane	0.2	Link	Link
Fenet v2	dla34	culane	0.1	Link	Link
Fenet v2	dla34	culane	0.05	Link	Link
Fenet v2	dla34	llamas	0.4	Link	Link
Fenet v2	dla34	llamas	0.2	Link	Link
Fenet v2	dla34	llamas	0.1	Link	Link
Fenet v2	dla34	llamas	0.05	Link	Link

This table showcases a couple of multi-task neural networks designed for autonomous driving. These models integrate lane detection, drivable area detection, and object detection. While they are capable of performing multiple tasks, they can also be used for only lane detection.

Model	Video-Campus	Video-Road
YOLOPv2	Link	Link
HybridNets	Link	Link
Twinlitenet	Link	Link

Based on my past experiences, I can say that these models yield better results when trained with our own data.

Additionally, these models perform better in highway-like environments, and the datasets used to train these models are constructed in highway-like environments. Therefore, tests conducted with highway data will yield better results. In addition to the results I've shared, I will also share the results obtained with data newly collected by me.

StepTurtle commented 1 month ago

We tested some different lane detection models and we don't think any of them looks enough. So if we want to use one of these models, we need to train these models with our labeled data. So, we close this task for now as won't do.

autowarefoundation / autoware