search

cfzd / Ultra-Fast-Lane-Detection

Ultra Fast Structure-aware Deep Lane Detection (ECCV 2020)
MIT License
1.82k stars 493 forks source link

Extract the detected lane pixel #301

Closed k-nayak closed 2 years ago

k-nayak commented 2 years ago

Hello @cfzd,

thank you for such amazing work.

I want to know if there are any methods to extract the array of detected lane pixels? As I want to try to calculate the lane curvature using the detected lanes.

Any approaches that I can use?

Thanks.

Glutamat42 commented 2 years ago

What you get back is basically a list of x,y coordinates on the current sample for every of the 4 lane markings (if there are less you'll get the rest class for x). If you want the "pixels" you can use the coordinates to address a specific pixel and extract it. But to calculate the curvature i would use the list of (x,y).

k-nayak commented 2 years ago

Thanks for the quick response @Glutamat42, I am mostly interested in just getting one or two lanes accurately and extracting the detections. As per my research, I will have to convert the pixel to meters to use in the curvature formula.

Glutamat42 commented 2 years ago

The project is designed to predict 4 lane markings. If you want you can change that, but i would just stick with it and ignore the predictions you are not interested in. If you want the "center" lane (the one your car is on atm) just use the 2nd and 3rd lane markings and you have the left and right border of the lane.

I can't really help you with the curvature calculation, but i if you cant avoid converting the coordinates on the picture to "real" meters you have to do it by yourself based on your specific camera and it's position. It's more or less a standard problem of computer vision and you'll find the formula for that. https://github.com/Glutamat42/Carla-Lane-Detection-Dataset-Generation is doing it the other way around (convert in-simulation coordinates to image coordinates)

k-nayak commented 2 years ago

basically, I am trying to find a way to integrate my code with https://github.com/mithi/advanced-lane-detection to get real-time lane curvature except the lane is now being detected using deep learning model. I will also look into how I can skip the part of converting from pixel to meters if I can get it working. Thanks for your valuable input @Glutamat42

k-nayak commented 2 years ago

Hello @Glutamat42, Hope you are doing great.

I had a model-related question. I am able to compute the curvature as intended using inverse perspective mapping of the detected lane coordinates (u,v) from lane 1 to (x,y) in meters. however, the function takes in these coordinates and computes the curvature after curve fitting. At the moment when the car in Carla is at an intersection, there exist no lane markings at the center of the junction and it fails to return any pixel coordinates as a result.

Which fails to return any curvature values as a result. Is there any way to overcome such an issue? Or any advice to maybe train the model in any other way to detect any pixels in the junction area?

Glutamat42 commented 2 years ago

ufld detects the lane markings to follow lanes. In case of crossings there are multiple possibilities. Without further information (like a navigation map with the desired path drawn on it or something like that) it is impossible for the model to predict the "path" because there are multiple possibilities.

Because of that we left crossings aside / scenes where a crossing is too near or the car is at a crossing were removed for our project. I'm not totally sure how culane/tusimple are handling this case, but i would guess there are also no or only a few crossings/intersections

k-nayak commented 2 years ago

I see so do you think adding data (more crossing images) from the cross-sections and training would help generalize the detection by any chance?

Also, was the model for your project trained on images from these sections?

I am currently facing the issue of computing curvature at the crossing and looking for alternatives to get it.

Thanks for the quick reply.

Glutamat42 commented 2 years ago

It doesn't matter how many samples you add. you can't predict the unpredictable. How should the model know which direction the car will take only from a camera picture. It might become possible when you are already halfway across the intersection, but if you are before or at the start of the crossing it is simply impossible to know for the model which of the multiple possible "lanes" you will take

Also, was the model for your project trained on images from these sections?

Like i mentioned, we cut the scenes if the car got too close to a crossing

k-nayak commented 2 years ago

Alright, that's very helpful thanks. Maybe some postprocessing, where curve extrapolation can help? It's just an idea at the moment.