Wrong label for Seq05VD_f02610_L.png

[hubutui]

Hi, I convert label from RGB mode to 'P' mode based on class_dict.csv, and found there are some points/coord with wrong label, aka its color is out of color defined in class_dict.csv. Here is how I convert RGB mode to 'P' mode and check label:

from PIL import Image
from utils import get_label_info

labelinfo = get_label_info(csvfile)
palette = []
for key in labelinfo:
    for item in labelinfo[key]:
        palette.append(item)

ref_image = Image.new(mode='P', size=(1, 1))
ref_image.putpalette(palette)

img_rgb = Image.open('rgb_mode.png')
img_p = img_rgb.quantize(palette=ref_image)

img_p_wrong = img_p[img > 31]
img_rgb_wrong = img_rgb[img > 31]

I'm not sure how to deal with this image, currently I simply remove it out of my training/val/testing. After convert RGB mode to 'P' mode, I don't need to use one hot encoding for label anymore. It seems that the one hot encoding & decoding slow down training speed.

And, the code you calculating accuracy seems weird, https://github.com/ooooverflow/BiSeNet/blob/master/utils.py#L103, pred & label are in shape of 1xCxHxW, where C is the channels, which is 3 in this case. pred[:, :, h, w] == label[:, :, h, w] means one pixel prediction right, not three.

[ooooverflow]

For question 1, I may update the code for accelerating the train without ont-hot encoding transformation, thanks for you suggestion anyway. For question 2, check the https://github.com/ooooverflow/BiSeNet/blob/master/utils.py#L78, colour_code_segmentation function will change the 1 x C x H x W to H x W x 3

[hubutui]

Yeah, maybe convert label mask to palette mode is better. And for calculating mIOU, metrics module from sklearn might helps a lot.

[ooooverflow]

Thanks for your suggestion! Many metric code in this model comes from GeorgeSeif's work, I will update them in the future