search

ika-rwth-aachen / MultiCorrupt

MultiCorrupt: A benchmark for robust multi-modal 3D object detection, evaluating LiDAR-Camera fusion models in autonomous driving. Includes diverse corruption types (e.g., misalignment, miscalibration, weather) and severity levels. Assess model performance under challenging conditions.
MIT License
41 stars 5 forks source link

Missing of the file for reproducibility #2

Closed Barath19 closed 4 months ago

Barath19 commented 4 months ago

Hi,

I am trying to run the example. However, I am not able to create this file ("deepinter_infos_val.pkl") or find it in the repo. I have the nuscenes_val_infos.pkl file for the nuscenes dataset, this does not work (key error). May be that, as mentioned in your paper, you are creating a new corrupted dataset and this .pkl file should be generated for that??

Reference for the dependent line of code. https://github.com/ika-rwth-aachen/MultiCorrupt/blob/57f91ba36b258e4d4b3428020d9479ac695c0b24/converter/img_converter.py#L41

TillBeemelmanns commented 4 months ago

Hi Barath19,

thank you for pointing this out! Probably I forgot the case that depending on the version of mmdetection3d there are different variants of .pkl files which are not compatible. I will try to find a solution this week.

BTW which mmdetection version are you using or how did you generate your nuscenes_val_infos.pkl file ?

Best, Till

Barath19 commented 4 months ago

Hi Till,

Thank you for the swift reply.

I have use the latest (v1.4) of the mmdetection3d to generate the .pkl file. https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/nuscenes.html

Best regards, Bharath

TillBeemelmanns commented 4 months ago

Hi Barath19,

I added support for .pkl files generated by mmdetection3d from the newer dev-1.x branch. Please checkout the latest release https://github.com/ika-rwth-aachen/MultiCorrupt/releases/tag/v0.0.5

Let me know if your problems have been resolved.

Best, Till

Barath19 commented 4 months ago

Thank you for the changes it works with the newer version of mmdetection3d. However, I had to make changes to https://github.com/ika-rwth-aachen/MultiCorrupt/blob/3fb6c287c5a2a158f6608a8b159b6d33e244b57b/converter/lidar_converter.py#L28

sweep path removing characters from 16 to 41 (based on my path).

TillBeemelmanns commented 4 months ago

Hmm okay, probably depends how you compile the .pkl file. What was the last part that you had to cut off ? e.g. [41:]

Barath19 commented 4 months ago

Yeah it was [41:].

TillBeemelmanns commented 4 months ago

Thanks, alright. I will further improve this path conversion by taking only the last parts of the original path.

Barath19 commented 4 months ago

Awesome, thank you very much for your work.