search

yuliangguo / Pytorch_Generalized_3D_Lane_Detection

[ECCV 2020] Official PyTorch implementation of 'Gen-LaneNet: a generalized and scalable approach for 3D lane detection'
307 stars 78 forks source link

Need of gt_height and gt_pitch for inference: GeoNet3D_ext #16

Open gautamjain1009 opened 2 years ago

gautamjain1009 commented 2 years ago

Hey! I have explored your implementation quite nicely. The GeoNet3D_ext model takes as an input the features from the lane segmentation model. Thus those features are then fed to the GeoNet3D_ext model for 3d lane detection. From your implementation, I have realized that you are not predicting camera height and camera pitch instead you are using the gt camera height and gt camera pitch for the same and in your training script you are updating your M_inv using the same gt_camera_height and gt_camera_pitch.

https://github.com/yuliangguo/Pytorch_Generalized_3D_Lane_Detection/blob/5bb190ed9f0dc211cd6265ab30c151d3699d3ed4/main_train_GenLaneNet_ext.py#L218

            if not args.fix_cam and not args.pred_cam:
                model2.update_projection(args, gt_hcam, gt_pitch)

I tried to run the train script for GeoNet3D_ext and try to found out that gt_cam_height == pred_cam_height and the same for pitch.

Whereas in the unofficial implementation of 3D lanenet as per the train script, the height and pitch are different from gt's.

Consider a real-world scenario where I want to use the pre-trained models provided by you for 3D lane Detection: in that particular case camera_height and camera_pitch are fixed as not really predicted by the model. Does in that particular case the results for 3d lane detection will differ? Have you tested your approach with any real-world data?

CompliceU commented 2 years ago

@gautamjain1009 Hi, have you tried training with OpenLane?

gautamjain1009 commented 2 years ago

WIP. I will train soon and will open source the code for that.

CompliceU commented 2 years ago

@gautamjain1009 That's great too, I'm currently trying this to work and am having a huge problem preventing it from training properly

CompliceU commented 2 years ago

@gautamjain1009 Hello, I don't know if you have tried it. I tried it. I feel that the effect is very different from the results provided by the author of openlane. During the training process, the loss of the verification part has not been effectively reduced.