search

hustvl / MapTR

[ICLR'23 Spotlight & IJCV'24] MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction
MIT License
1.08k stars 163 forks source link

Av2 data curruption #83

Closed imyyf closed 1 year ago

imyyf commented 1 year ago

https://github.com/hustvl/MapTR/blob/11307ed835dd7534ebb8927df9609f0b1b825aa6/projects/mmdet3d_plugin/datasets/av2_map_dataset.py#L1202 https://github.com/hustvl/MapTR/blob/11307ed835dd7534ebb8927df9609f0b1b825aa6/projects/mmdet3d_plugin/datasets/av2_map_dataset.py#L1203 https://github.com/hustvl/MapTR/blob/11307ed835dd7534ebb8927df9609f0b1b825aa6/projects/mmdet3d_plugin/datasets/av2_map_dataset.py#L1203 Have you met Av2 data curruption? We meet the image data is none or the lidar data is none. In our Av2 data, there are 4% val data are curruption. Does it influence the evaluation rationality？

LegendBC commented 1 year ago

All models are evaluated under the same evaluation data. Though there exists currupted data, we don't think it will affect the fariness of the valuation. The only incurred issue is that it will decrease the samples for evaluation, but AV2 data frequency is much higher than nuScenes (10Hz vs 2Hz). So the number of evaluated samples are enough compared to nuScenes. In conclusion, we don't think it will influence the evaluation rationality.

imyyf commented 1 year ago

Thanks for your reply