search

VinAIResearch / ISBNet

ISBNet: a 3D Point Cloud Instance Segmentation Network with Instance-aware Sampling and Box-aware Dynamic Convolution (CVPR 2023)
Apache License 2.0
104 stars 22 forks source link

Performance improvement gap between the val and test set on ScanNet #23

Closed SARIHUST closed 10 months ago

SARIHUST commented 1 year ago

Hi, thanks for your great work on 3D instance segmentation. I have a question about the performance on the ScanNet dataset.

I noticed that your results on the test set improved by 3.2% on the AP50 metric compared with the val set results. However, when I tried my own model on the ScanNet benchmark, I found that my val set and test set results are very similar. Do you know why ISBNet performs much better on the test set? I would love to know how to improve my own models.

Looking forward to your reply, thanks in advance!

ngoductuanlhp commented 1 year ago

I guess one reason is that the official test set comprises only 100 scenes, leading to more unstable results.