Really bad inference results

[sowmen]

The inference outputs from the model are really bad even for very easy images.

1. Using Laneatt_Res18_Culane

2. Using SCNN_Res50_Culane

Any idea why this is happening? I've just done normal inference without any changes.

[Turoad]

Have you checked this issue https://github.com/Turoad/lanedet/issues/4 ?

[ibaiGorordo]

Hi, thank you for making this repository.

I have been doing some testing, and it actually is a big problem for detecting lanes on images that are not in the correct format. Because the values in sample_y have to also be considered, and modifying the values in sample_y bring some problems.

The simplest but suboptimal solution is to add a resize in the preprocess function (in detect.py) to make sure that the image matches the expected size.

def preprocess(self, img_path):
      ori_img = cv2.imread(img_path)
      ori_img = cv2.resize(ori_img, (self.cfg.ori_img_w, self.cfg.ori_img_h)) # <-Add this
      img = ori_img[self.cfg.cut_height:, :, :].astype(np.float32)
      data = {'img': img, 'lanes': []}
      data = self.processes(data)
      data['img'] = data['img'].unsqueeze(0)
      data.update({'img_path':img_path, 'ori_img':ori_img})
      return data

Otherwise, the logic in lane_seg.py of how sample_y is used might need to be modified.

[Turoad]

Thanks for your interset.

I guess the image you tested is not in the corresponding dataset, e.g., test a image which is not in CULane dataset with the model trained in CULane dataset. If yes, I believe the inference result may not be satisfying. It's better to train the model on a new dataset and redesign the sample_y. One dataset should have the corresponding sample_y.

The suggestion you provided is helpful. Maybe the sample_y can be other format, e.g., the normalized coordinate. I may update it in the next version.

[ibaiGorordo]

Yes that is correct. I think this repository is very interesting because it allows to test different models without having to download/set up different repositories. So, I would love it it was possible to do inference with data outside those datasets (videos, images and even webcam).

Yesterday I also was checking about the logic of sample_y. I have not looked much into the probability map, but currently the main purpose of sample_y is to have the lane points separated every N pixels (20 in Culane and 10 in TuSimple) in the original image, right?

Since the probmap2lane function anyway returns the normalized coordinates, sample_y could be defined with something like this in lane_seg.py:

sample_y = np.arange(0.99,0,-0.01)) 
for y in sample_y:
    proj_y = int(y*self.cfg.img_height)

And then in, to_array() function you would have to add the effect of the cut_height after you convert the points to the original image axis.

I have not tested it, so I am probably missing something. Also, it might be nice also if the original image size could be directly obtained from the input image during inference instead of having to set it up at the beginning.

[Turoad]

Thanks again. It is my pleasure that this repository can help you. Yet, it seems this repository still has deficiency. Welcome for PRs.

Yesterday I also was checking about the logic of sample_y. I have not looked much into the probability map, but currently the main purpose of sample_y is to have the lane points separated every N pixels (20 in Culane and 10 in TuSimple) in the original image, right?

That's right. Currently, 10 pixels is the setting of Tusimple' GT, 20 pixels in CULane follows the setting of SCNN. The sample_y may not simply be adjusted as np.arange(0.99,0,-0.01)). The sample pixels may inference the final accuracy.

So I think it might be the following format:

sample_y = np.arange(up, down, sample_pixels) / img_height

[ibaiGorordo]

True, I was only thinking about the inference, but in your case, you also want to have the lane points defined at specific heights to calculate the accuracy.

Actually, if it is okay I might create a separate repository focused only on inference with a simplified version of this repository with only the necessary elements (I would like to remove the mmdetection part but it is probably hard). However, I am not sure when I will have time though.

[Turoad]

Do you mean you want to create a repo due to the logit of sample_y? I don't really understand why you want to do so.

I think sparse lane points (defined at specific heights) is enough for inference or visualization. If you wan't to dense points, just interpolate.

Currently, I doesn't have the dependency of mmdetection but only mmcv. It's easy to add CondLaneNet with mmcv. I also want to remove the mmcv later.

[ibaiGorordo]

When I try a repository like this, I like to simplify it, removing the parts not necessary for inference, so that I can understand better what the model is doing (for example in the ultrafast lane detection, only 2 script were necessary). So, I was thinking to do something like that here.

But if you prefer not to, I can keep it privately without problem.

[readerrubic]

The repository is a good work, but it's hard for reader with bad programming ability to understand what the model is doing as @ibaiGorordo said. @ibaiGorordo you did a good job for simplifying ultra fast lane detection model, which helps me to absolutely understand ufld, I like it.