Closed leoli646 closed 4 months ago
Thanks for your interest!
In our experiments, we didn't tune the seeds . I've just re-cloned my repository code and re-ran the experiment with the newest configuration and --seed 0 for Modelnet40 & scanobjnn_hardest, obtaining the same results as in the paper. |
seed | results | log | |
---|---|---|---|---|
Scan_hardest | 0 | 85.08 | 20240327_112141.log | |
ModelNet40 | 0 | 93.48 | 20240327_112744.log |
I will update the configuration parameters and training log in the README.md file, and you can retry it later. You can also test it by downloading the weights provided in the README.md.
We have also noticed that different hardware platforms (GPUs, etc.) can impact the experimental results (even different GPU models on the same device may produce different results), probably because of variations in how certain arithmetic operations are conducted, which is a common occurrence. Hence, all experimental results are achieved by a single RTX 4090.
Thanks to the great work! I am really interested in this work. However, I have encountered some difficulties while attempting to reproduce the results on Modelnet40, scanobjnn_bg & scanobjnn_hardest (w/o voting).
I've attached an image depicting the results I obtained. As you can see, the performance of the baseline (pointmae) closely matches the reported results. However, when using pointmae+dapt, there is a significant deviation from the reported values.
I conducted the experiments using a single A100 GPU with default seed value of 0 and default data_aug (PointcloudScaleAndTranslate), unless otherwise specified.
I eagerly await your response and guidance on this matter.