search

Haiyang-W / CAGroup3D

[NeurIPS2022] This is the official code of "CAGroup3D: Class-Aware Grouping for 3D Object Detection on Point Clouds".
93 stars 9 forks source link

Reproduce the results on SUN-RGBD #6

Closed EternityDS closed 1 year ago

EternityDS commented 1 year ago

Hi,

Thanks for your great work! I follow the instruction and try to reproduce the results on SUN-RGBD. But I only get 61.18 mAP0.25 and 45.14 mAP0.5. These numbers seem much lower than the reported numbers. May I ask what I did wrong? I attached my training log here for your reference. Thanks! log_train_20230219-210950.txt

pigtigger commented 1 year ago

Hi, This result (61.18@mAP0.25) is very strange, and I wonder if your data has some bugs. Did you create the data by yourself or use the processed data in this repo? My previous SUN-RGBD pcdet training logs were unfortunately cleaned but I found some screenshots when re-implementing CAGroup3D on pcdet. 98791677047376_ pic (Note that the performance of SUN-RGBD fluctuates greatly.) Btw, could you retrain the model with 8x2 because I find your training config is 1x8? And I will also train a SUN-RGBD model and attach the logs and weights asap.

Haiyang-W commented 1 year ago

Hi,

Thanks for your great work! I follow the instruction and try to reproduce the results on SUN-RGBD. But I only get 61.18 mAP0.25 and 45.14 mAP0.5. These numbers seem much lower than the reported numbers. May I ask what I did wrong? I attached my training log here for your reference. Thanks! log_train_20230219-210950.txt

You can first retrain the model with 2x8 (batch size 8 per GPU and two GPU cards in total) because I find your training config is 1x8 ? Our log and pretrain-weight will be released as soon as possible. I am also a PKUer and now rushing the some DDLs. If necessary, you can email and discuss with me in person. Hope that can ease your concerns.

EternityDS commented 1 year ago
image

Thank you all for the quick reply! I used the processed data in the repo link. I can reproduce the result on ScanNet but get trouble on SUN RGB-D. I tried another experiment with 16x1 batch size and I attached my log here. I will try 8x2 soon.

"If necessary, you can email and discuss with me in person." --- That would be great! Can I get your email?

Haiyang-W commented 1 year ago
image

Thank you all for the quick reply! I used the processed data in the repo link. I can reproduce the result on ScanNet but get trouble on SUN RGB-D. I tried another experiment with 16x1 batch size and I attached my log here. I will try 8x2 soon. "If necessary, you can email and discuss with me in person." --- That would be great! Can I get your email?

Perhaps it's something else, as adjusting the batch size doesn't appear to be the solution. My email is "wanghaiyang6@stu.pku.edu.cn". If you have any questions, please feel free to contact me. :)

Have a good day! Haiyang

Haiyang-W commented 1 year ago
image

Thank you all for the quick reply! I used the processed data in the repo link. I can reproduce the result on ScanNet but get trouble on SUN RGB-D. I tried another experiment with 16x1 batch size and I attached my log here. I will try 8x2 soon. "If necessary, you can email and discuss with me in person." --- That would be great! Can I get your email?

Have you reproduced it on your side? Let me know if you have any questions. Thanks! :)

EternityDS commented 1 year ago

Hi,

Really thank you for keep following up! I have reproduced the result with 8x2 batchsize setting but still cannot reproduce with 16x1 or 8x1. I was working on other things and did not look into this issue. Have you tried to reproduce the result with 1 GPU?

Haiyang-W commented 1 year ago

Hi, Sorry for that, I've only tried the 8x2. In principle, 16x1 should be fine. I'll try this setting later.

Haiyang