search

mit-han-lab / spvnas

[ECCV 2020] Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution
http://spvnas.mit.edu/
MIT License
587 stars 109 forks source link

reproduce result low #94

Closed yjumpzhighy closed 2 years ago

yjumpzhighy commented 2 years ago

Hi,

When I run evaluate.py using pretrained 'SemanticKITTI_val_SPVCNN@119GMACs' without making any changes to the repo, and check the miou results, I found that some class has poor performance.
Like the motorcyclist shows almost 0% miou, while the paper says about 40% miou? Anyone know why?

kentang-mit commented 2 years ago

Hello @yangzuyuanhao,

In the paper we only report the test set results. I remember that the val set contains very few samples for motorcyclist thus the performance is indeed low for our method.

Best, Haotian

yjumpzhighy commented 2 years ago

Hello @yangzuyuanhao,

In the paper we only report the test set results. I remember that the val set contains very few samples for motorcyclist thus the performance is indeed low for our method.

Best, Haotian

Got it, thanks!