search

Pointcept / PointTransformerV2

[NeurIPS'22] An official PyTorch implementation of PTv2.
356 stars 25 forks source link

How to improve the test performance on ScanNet? #34

Open zymhnu opened 1 year ago

zymhnu commented 1 year ago

Thanks for sharing. I had some problems when I submitted my testing results to ScanNet. I had a competitive performance on the validation set, such as 74.0% mIoU, but only had a low performance on the test set, like 71.0%mIoU. Moreover, I reproduced ptv2 and got 74.5% mIoU on the validation set, and I only got 71.1%mIoU when submitted to ScanNet.

haibo-qiu commented 1 year ago

Hi, I share the same concern as I achieved 74.3% on the ScanNet validation set but only managed to obtain 70.6% on the ScanNet test set.

Gofinge commented 1 year ago

I never encounter the same situation. I submitted the benchmark twice, one for 73.x and another is 75.x. We also discussed the phenomenon, and some senior researchers told me that the test result on the ScanNet benchmark is not stable, and the only solution is to try more time.

zymhnu commented 1 year ago

Thank you for your reply. But are there any tricks on the ScanNet test? Or just modify the value of "split"?