search

Fsoft-AIC / Open-Vocabulary-Affordance-Detection-in-3D-Point-Clouds

[IROS 2023] Open-Vocabulary Affordance Detection in 3d Point Clouds
https://openad2023.github.io
MIT License
54 stars 1 forks source link

Don't know what cause the low IoU problem #8

Closed Tz2H closed 5 months ago

Tz2H commented 7 months ago

Hi! Thanks for your open source. But when I trying to reproduce it, my result has a quite low IoU value. I didn't change any value in the training cfg except the root to data. Could you give me some help? That would be wonderful!

屏幕截图 2024-04-22 195441
toannguyen1904 commented 5 months ago

Hi @Tz2H, Sorry for the late response. As I see your result is quite reasonable. The overall metric outcomes are comparable to those of our report. Perhaps you should keep training for more epochs. In fact, we report our results as the average of 3 different seeds of 0, 1 and 2, so your one-time result might be not the same as ours.

For the IoU of some affordance labels are zeros, your outcome is the same as ours. That is because of the way we zero-shot test it, in which we feed all unseen labels at once. This makes some labels dominate others. If you want to see more comprehensive results on how the model performs on each affordance label, you can try feed to the model that affordance and only the background one “none”.

Best regards, Toan.