search

OpenGVLab / InternImage

[CVPR 2023 Highlight] InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions
https://arxiv.org/abs/2211.05778
MIT License
2.49k stars 232 forks source link

Problem to reproduce OpenLanev2 result #214

Open hagianga21 opened 1 year ago

hagianga21 commented 1 year ago

Hi, I am trying to reproduce the reported results on OpenLanev2. I didn't change anything. I downloaded the checkpoint and run the test script with this cmd: ./tools/test.sh plugin/mmdet3d/configs/internimage-s.py internimage-s-openlane.pth 8 --eval * --eval-options dump=True dump_dir= work_dirs

But the performance is not that good. Here is the results I got: {'DET_l': 0.0005530441, 'DET_t': 0.002188278, 'TOP_ll': 0.0, 'TOP_lt': 6.238920963764743e-05, 'OpenLane-V2 Score': 0.0026600015060585045, 'F-Score for 3D Lane': 0.0033523358535203166}

Could you please help me take a look? Thanks

cyty98 commented 1 year ago

Got a similar result. Did you figure out what's the problem? {'DET l':0.031355385, 'DET t':0.0020301112, 'TOP ll': 5.200495277053748e-05, 'TOP lt': 0.0009464013969901333, 'OpenLane-V2 Score': 0.017840144855253124, 'F-Score for 3D Lane': 0.13003625322048777}

hagianga21 commented 1 year ago

No, I didn't. Have you tried training it instead of using the checkpoint? Maybe something wrong in the checkpoint

cyty98 commented 1 year ago

Not yet, that is going to take some time. Btw I noticed that the config file they provided is not completely coherent with the checkpoint file. In autonomous_driving/openlane-v2/plugin/mmdet3d/configs/internimage-s.py line23, the number of 3D control points for each lane curve is set to 5, yet the checkpoint file contains parameters for a detector head which produces 11 points. (cf figure below) image I got my result by correcting this mistake in config, but as you see, it's still far from their result. Maybe there are other mistakes which lead to this poor performance?