search

huixiancheng / CENet

[ICME 2022] CENet: Toward Concise and Efficient LiDAR Semantic Segmentation for Autonomous Driving
MIT License
99 stars 13 forks source link

Question about showing different result for the validation set #26

Closed CK-Sung closed 1 year ago

CK-Sung commented 1 year ago

Hi, Thank you for great work!

I have question about the mIOU result.

When I trained the model, the best mIOU for the validation set is about 64. But, When I infer using the SENet_valid_best model, then evaluate the validation set using the semantic_kitti_api & evaluate_iou.py, it shows 61. (64 x 512)

Is there any mistake? or it is as usual?

Thank you

huixiancheng commented 1 year ago

It is normal. The validation results are calculated on 2D during training, and there will be losses when projected back to 3D points, so the mIoU will be a little lower, and the lower the resolution, the more it will fall off a little more.

In addition, the performance on the test set is usually slightly lower than on the validation set. The performance on the test set can be improved by fine-tuning the validation set. This is a trick.

CK-Sung commented 1 year ago

Thank you! I understand it :)

Hayleycat commented 1 week ago

Thank you! I understand it :) Hello, I am very confused. Do you know how the author obtained the results of the ablation experiment. I see from the results given by the author that the maximum training result is only 63.5miou, how did the original text get 65.3 miou. Is there any other way to obtain the results of ablation research? Can you please let me know? Thank you.