How to compute the OA and mIoU on the Semantickitti dataset

[longmalongma]

How do you compute the OA and mIoU on the Semantickitti dataset? You don't seem to have published the code for it.

[yanx27]

@longmalongma Hi, when you submit to official semanticKITTI benchmark, it will generate output log for OA and mIoU. Also, if you want to compute OA and mIoU for validation set, you can use official semanticKITTI api here just by following codes:

$ ./evaluate_semantics.py --dataset /path/to/kitti/dataset/ --predictions /path/to/method_predictions --split train/valid/test # depending of desired split to evaluate

By running test_semantic_kitti_grid.py, it will automatically generate predicted labels for evaluation.

[longmalongma]

@longmalongma Hi, when you submit to official semanticKITTI benchmark, it will generate output log for OA and mIoU. Also, if you want to compute OA and mIoU for validation set, you can use official semanticKITTI api here just by following codes:

$ ./evaluate_semantics.py --dataset /path/to/kitti/dataset/ --predictions /path/to/method_predictions --split train/valid/test # depending of desired split to evaluate

By running test_semantic_kitti_grid.py, it will automatically generate predicted labels for evaluation.

Thanks for your reply, I see it!

[longmalongma]

@longmalongma Hi, when you submit to official semanticKITTI benchmark, it will generate output log for OA and mIoU. Also, if you want to compute OA and mIoU for validation set, you can use official semanticKITTI api here just by following codes:

$ ./evaluate_semantics.py --dataset /path/to/kitti/dataset/ --predictions /path/to/method_predictions --split train/valid/test # depending of desired split to evaluate

By running test_semantic_kitti_grid.py, it will automatically generate predicted labels for evaluation.

Thanks for your reply, I see it!

Can it compute OA and mIoU for test set???

[yanx27]

@longmalongma You need to submit on official benchmark of semanticKITTI

[longmalongma]

Thanks for your reply, can you tell me which files ｓhould I submit? only .label files from sequences 08,11 to 21? Can you give me the web site to refer to the predicted results？　Ｓorry, I am first use senmantickitti daatset.

[yanx27]

Hi, @longmalongma , sequence 08 only for validation set. If you want to submit to official semantickitti benchmark here, you need to firstly generate sequence 11~21. Then zip them together and submit to the single scan competitions.

[longmalongma]

Hi, @longmalongma , sequence 08 only for validation set. If you want to submit to official semantickitti benchmark here, you need to firstly generate sequence 11~21. Then zip them together and submit to the single scan competitions.

Thanks, Ｉs only .label files it need?