dongkwonjin
commented
4 months ago As you pointed out, ILD may yield better performances in the image metrics, compared to PLD.
However, in terms of video metrics, such as flickering and missing rates, ILD yields inferior results than PLD.
In other words, PLD provides temporally more stable detection results of lanes than ILD.
Thank you for your attention.
When I evaluated the authors' provided ILD_Coeff model, I did not get the F1 score of 0.787 reported in table 4 of the paper. Instead I got F1 score of 0.8056.
This is the exact output of the eval:
precision 0.7053766470275624, recall 0.5420338264165887, fscore 0.6130106817959742
culane laneatt metric evaluation start!
Loading prediction data... 100%|████████████████████████████████████████████████████████████████████████| 23021/23021 [00:08<00:00, 2827.61it/s] Loading annotation data... 100%|████████████████████████████████████████████████████████████████████████| 23021/23021 [00:09<00:00, 2302.34it/s] Calculating metric in parallel... 100%|█████████████████████████████████████████████████████████████████████████| 23021/23021 [02:50<00:00, 134.98it/s] ====================Results (datalist.txt)==================== TP: 62200 FP: 13849 FN: 16172 Precision: 0.8179 Recall: 0.7937 F1: 0.8056
culane laneatt metric evaluation done!
But I do get the authors' reported F1 score when I evaluate their provided PLD model, with an F1 score of 0.8249 which matches the paper's number in table 4 of 0.825.
This is the exact output of the eval for PLD: precision 0.6302282598102822, recall 0.5900906007422002, fscore 0.6094992939678525
culane laneatt metric evaluation start!
Loading prediction data... 100%|█████████████████████████████████████████| 23021/23021 [00:07<00:00, 2891.16it/s] Loading annotation data... 100%|█████████████████████████████████████████| 23021/23021 [00:10<00:00, 2285.51it/s] Calculating metric in parallel... 100%|██████████████████████████████████████████| 23021/23021 [02:47<00:00, 137.51it/s] miou ===> 0.7256893668357335 ====================Results (datalist.txt)==================== TP: 64516 FP: 13541 FN: 13856 Precision: 0.8265 Recall: 0.8232 F1: 0.8249
culane laneatt metric evaluation done!
I am wondering if this is expected or if I did something wrong? Because this is not as much of an improvement from single-frame to video-based detection as is reported in the paper.
Thanks.