search

liuruijin17 / LSTR

This is an official repository of End-to-end Lane Shape Prediction with Transformers.
BSD 3-Clause "New" or "Revised" License
647 stars 130 forks source link

model's robostness is so bad? #27

Closed cutefulf closed 3 years ago

cutefulf commented 3 years ago 
I run the demo on TuSimple test data by following the master branch(Tusimple) readme.md with the default model you offered. the result is just ok. but when i use the model to test my own test data or the Culane test data, the performance is very bad. so i changed to the Culane branch to make a test(use culane model) with Tusimple data or my own data, the result is also very bad.
I knew that different data were captured by different cameras. so different data related to different camera params. I don't know if  the model's polynomial coefficients should be aligned with the test data's camera params when testing with different dataset?
liuruijin17 commented 3 years ago

Sorry for being late, tough months about lots of deadlines.

We now provide two methods about how to play with other datasets.

Two methods:

No training and directly use pre-trained models 1-1 TuSimple Make sure the aspect ratio of your image is 360 to 640, then our test protocol resizes your input image to 360 x 640. 1-2 CULane Make sure the aspect ratio of your image is 295 to 820, then our test protocol resizes your input image to 295 x 820. 1-3 Why not good? Currently, images in TuSimple contain only highway scenes, and the quantity is small. Another CULane contains urban scenes a lot but still suffers the lack of lane shape pattern (curved, closely curved, remotely curved, ...). 1-4 How to improve? Crop (or any other transformations) your images to make them have similar shape patterns like CULane (or TuSimple) since we treat each lane as a whole object.

Annotating your image and training 2-1. How to annotate? (Some recommendations) 1). Annotate each lane with keypoints (10 ~ 20 points is ok, you can sparsely label the close part and densely label remote curved part for better performance on remote curved structures). 2). Make sure the occluded part is correctly imaged when you annotate a lane 3). Make sure the heavily occluded lane is labeled as ignored 4). 2) and 3) are important to form consistent shape guidance, which avoids the problem of marker-appearance misalignment brought about by arbitrary markers. 5) 2), 3) and 4) are found based on the fact that we treat each lane as a whole object (like a cat or dog in the object detection field) rather than low-level pixels. 6) Remember and think 5), you will make your mind about the advantages and disadvantages of LSTR. 2-2. Big gains 1) Adding channels in the backbone, at least 4% CULane F1 improvement. 2) Pre-train your model on a larger dataset, e.g. pre-trained on LLAMAS can bring at least 0.2% TuSimple accuracy improvement.