search

Pointcept / PointTransformerV3

[CVPR'24 Oral] Official repository of Point Transformer V3 (PTv3)
MIT License
804 stars 47 forks source link

Performance gap in Test sets of Scannet vs. Scannet 200? #114

Open TerenceWallel opened 2 days ago

TerenceWallel commented 2 days ago

Wonderful job. My collaborators and I have found that using PTV3'S code with the weights file, the model performs consistently in the Val set of Scannet and Scannet200, but there is a serious performance degradation in the Test set of Scannet and Scannet200 (over than 3% in Scannet and Scannet200).

We would like to ask if this performance degradation is normal? What steps do we need to follow to get points in the Test set that are consistent with the paper results?

TerenceWallel commented 2 days ago

According to the previous issue #25, I'm guessing that the test set metrics in the paper use more trick or TTA compare with Val set of Scannet. In order to further replicate PTv3, I'm wondering if it would be possible to provide a list of tricks for the test set using in the paper, as well as the associated code implementation?