unmannedlab / RELLIS-3D

RELLIS-3D: A Multi-modal Dataset for Off-Road Robotics
https://unmannedlab.github.io/research/RELLIS-3D
287 stars 49 forks source link

Cannot match reported SalsaNext performance with the released model #43

Open chahyon-ku opened 5 months ago

chahyon-ku commented 5 months ago

Hi authors! I've been trying to reproduce the SalsaNext results you have on your paper using the provided pretrained model (https://drive.google.com/file/d/1DxuzlnFKnU8EpSuODRywJUrJlieUUheg/view?usp=sharing). rellis3d_table reproduction My confusion comes from the fact that I am getting identical IOUs for the latter 11 classes, but much lower IOUs for the first 3 classes (grass, tree, bush), reducing the total mIOU from 43.07 to 37.8. I see that the table you have on your README also suffers from this: all other IOUs are identical to the ones on the paper except the first 3 classes, reducing the mIOU from 43.07 to 40.20. Do you guys know why this might be? Was the dataset/labels updated after you got your results? Thanks for the help!