search

qubvel / segmentation_models

Segmentation models with pretrained backbones. Keras and TensorFlow Keras.
MIT License
4.73k stars 1.03k forks source link

mIoU with background in semantic segmentaion #554

Open Gil-TakKong opened 1 year ago

Gil-TakKong commented 1 year ago

Hi!

I have a question about background in the calculation of the mIoU.

When we calculate the mIoU of the segmentaion, usually IoU of the background is included.

In that case, mIoU is at least 0.45 even the model didn't catch anything because of the IoU(background).

I wondering about Is the calculation(mIoU with background) correct or reasonable? If the calculation is not reasonable, then how can we measure the performance of the segmentation model?

Thank you.

jaswindersingh2 commented 1 year ago

Hi @Gil-TakKong

I think, background can ignored by passing class_indexes argument to the sm.metrics.IOUScore() function.

For example, to ignore background during IoU evaluation (for 3 classes CLASSES = ['sky', 'building', 'pole', 'unlabelled/background']), sm.metrics.IOUScore() function can be modified as follows:

metrics = [sm.metrics.IOUScore(threshold=0.5,class_indexes=[0,1,2]), sm.metrics.FScore(threshold=0.5,class_indexes=[0,1,2])]

Cheers