The file cls_labels.npy saves the class label of each image, format like {'filename' : '1, 0, 1, ..., 0'}. For simplicity, you can use the label from the x_mask, I think the program may seems like

[tong1101]

The file cls_labels.npy saves the class label of each image, format like {'filename' : '1, 0, 1, ..., 0'}. For simplicity, you can use the label from the x_mask, I think the program may seems like y = multi_hot(set(np.array(x_mask).tolist())) In general, the class label is not necessary for semantic segmentation problem, just suit yourself :)

Originally posted by @JaveyWang in https://github.com/JaveyWang/Pyramid-Attention-Networks-pytorch/issues/1#issuecomment-448924243

[tong1101]

I want to use the class label,and I don't know how to. As you say:The file cls_labels.npy saves the class label of each image, format like {'filename' : '1, 0, 1, ..., 0'}. The problem is that I still don't know how to create the file cls_labels.npy. For pascal voc datasets,the format like{'filename' : '1, 0, 1, ..., 0'},'filename' means all the images in the datesets? And what do 0 and 1 mean?

[Chenfeng1271]

I don't use this label and loss correspond to it. The result is 75.6%, what is your result of it?

---

The class label loss wouldn't affect the accuracy. If you wanna get higher accuracy, please set the image size to 512*512, and set the batch size as large as you can, at least 8 batch size. My experimental result is up to 78% mIOU.

--JaveyWang

[JaveyWang]

The class label file is a dict which saves the file name and multihot label like {'filename1' : [1, 0, 1, ..., 0], 'filename2' : [0, 1, 1, ..., 0], ......}, the '0' or '1' indicate whether the corresponding label exists in the image.

[zhenmafan7]

can u share the cls_labels.npy file? or where can i find it?

[JaveyWang]

can u share the cls_labels.npy file? or where can i find it?

Sorry for the inconvenience, I am not in the lab for now. If you wanna use the class labels, you can create it from the mask by yourself. see Issue.

[vipul0805]

Hi, I am also trying to implement PAN and after process the original SegmentationClass files and producing the raw segmented image files, which should have each pixel value between 0 : 20. But when I read the image from SegmentedClassAugRaw and converted x_mask to list it has all the pixel values as 0. can you tell me what am I doing wrong. when reading the image from SegmentedClass and mapping the pixel values to index of colormap I get an error because there are pixels in image with value [224, 224, 192] which is not in the colormap provided in the get_pascal_labels() function in utils.py of code provided by Chenfeng1271. acn anyone tell me what am I doing wrong here.

Here is what I did:

path="C:/Users/vj0805/Desktop/models-master/models-master/research/deeplab/datasets/pascal_voc_seg/VOCdevkit/VOC2012"
name='2007_003917'
x_mask = imread(path + '/SegmentationClass/' +name+ '.png')
pixels=np.array(x_mask[0]).tolist()
for val in x_mask[1:]:
    pixels.extend(np.array(val).tolist())
map = [[128, 0, 0], [0, 128, 0], [128, 128, 0],
                       [0, 0, 128], [128, 0, 128], [0, 128, 128], [128, 128, 128],
                       [64, 0, 0], [192, 0, 0], [64, 128, 0], [192, 128, 0],
                       [64, 0, 128], [192, 0, 128], [64, 128, 128], [192, 128, 128],
                       [0, 64, 0], [128, 64, 0], [0, 192, 0], [128, 192, 0],
                       [0, 64, 128],[0, 0, 0]]
lables=[]
for pixel in pixels:
    labels.append(map.index(pixel))

I get the following error: )

in () 1 lables=[] 2 for pixel in pixels: ----> 3 labels.append(map.index(pixel)) ValueError: [224, 224, 192] is not in list 

[JaveyWang]

@vipul0805 It seems that you have mishandled the original SegmentationClass files. Value [224, 224, 192] should not appear in the images. And there is a typo in your code:

lables=[]  # labels = []?!
for pixel in pixels:
    labels.append(map.index(pixel))

[vipul0805]

Thank you for your reply. I checked again by extracting the voc.zip and loaded the same image from that dataset and it is still showing the same [224, 224, 192] value appearing in the mask and I found a post which says that this pixel value corresponds to the void label i.e total of 22 labels instead of 21. I have also attached some images from the dataset in which this void pixel is appearing along with the link to the post: https://gist.github.com/wllhf/a4533e0adebe57e3ed06d4b50c8419ae

Thanks for pointing out the mistake with the labels array. So in the cls_labels.npy the dict, for each file_name as key the value should be a array of 0 and 1 of length same as number of actual labels present in the dataset i.e. 21?

[vipul0805]

@JaveyWang Can you please check whats wrong with it. Do I have to consider the void label or not and does the cls_labels.npy contains a dict, in which each file_name as key the value should be a array of 0 and 1 of length same as number of actual labels present in the dataset train_aug.txt i.e. 21?

[vipul0805]

@JaveyWang Color_classifier function is missing from your networks.py and because of that an error is coming while importing eval.py. Can you please tell me about implementing Color_classifier() function.

[JaveyWang]

@vipul0805

Q1: Actually, I haven't met the 22 labels problem in my VOC dataset. I think you could replace the void label with the [0, 0, 0] if the void label can be treated as background.

Q2: Yes, cls_labels.npy should contains multihot array of each image.

Q3: Sorry for the ambiguous. In fact, the 'Clour Classifier' part is not important, you can remove the relevant code for evaluation.

[vipul0805]

I apologize for bugging you continuously but I had one other doubt which is that you have trained your model on 10,582 Images but which are listed in train_aug.txt but there are no labelled images for those images so how did you get the ground truth labels for training data. Even after data augmentation the images I got in folder SegmentationClassAugRaw had just color palette removed from them.

[JaveyWang]

@vipul0805

I wonder if that you haven't correctly downloaded the VOC dataset. I am unable to solve your problem because I didn't experience the problem you met.

Please refer to the code in dataset.py:

x_mask = imread(self.data_path + '/SegmentationClassAug/' + self.names[index] + '.png', mode='L')
        x_mask = Image.fromarray(x_mask)

which works well in my case. :)

[vipul0805]

@JaveyWang

Thanks I got that part. I am facing another problem now which is as follows: I have 17,125 images in my JPEG folder and 12,031 in SegmentationClassRaw folder and while running the train.py I am getting KeyError: which is the getitem() func cannot find the image in the specified datapath and it is giving error for a different file each time i run the train.py. But I checked that for the images I am getting the error are present in the data folder. I cannot figure out why I am getting this error even when the file is present in the Jpeg and SegClass folder.

And are the number of images in JPEGImages and SegmentationClassAug are supposed to be equal or they supposed to be equal to the number i specified above.

[wszjzhang]

The class label file is a dict which saves the file name and multihot label like {'filename1' : [1, 0, 1, ..., 0], 'filename2' : [0, 1, 1, ..., 0], ......}, the '0' or '1' indicate whether the corresponding label exists in the image.

why not just upload this file and save everybody some time... thanks...

[zhudahui]

name 'multi_hot' is not defined,what should I do.

[zhudahui]

@JaveyWang name 'multi_hot' is not defined,what should I do.

[JaveyWang]

The class label file is a dict which saves the file name and multihot label like {'filename1' : [1, 0, 1, ..., 0], 'filename2' : [0, 1, 1, ..., 0], ......}, the '0' or '1' indicate whether the corresponding label exists in the image.

why not just upload this file and save everybody some time... thanks...

I am very sorry for that, I didn't check the code when I first upload this repository, this is my fault, and now I am not in the lab. The class label is NOT important for segmentation task, you can feel free to remove the relevant part. And everyone want to use the class label please generate the multi-hot label by yourself, this is a multi-label classification problem.

[zhudahui]

Thank you very much！

------------------ 原始邮件 ------------------ 发件人: "jiawei.wang"<notifications@github.com>; 发送时间: 2019年12月12日(星期四) 晚上6:27 收件人: "JaveyWang/Pyramid-Attention-Networks-pytorch"<Pyramid-Attention-Networks-pytorch@noreply.github.com>; 抄送: "我很怪丶"<1940078219@qq.com>; "Comment"<comment@noreply.github.com>; 主题: Re: [JaveyWang/Pyramid-Attention-Networks-pytorch] The file cls_labels.npy saves the class label of each image, format like {'filename' : '1, 0, 1, ..., 0'}. For simplicity, you can use the label from the x_mask, I think the program may seems like (#4)

The class label file is a dict which saves the file name and multihot label like {'filename1' : [1, 0, 1, ..., 0], 'filename2' : [0, 1, 1, ..., 0], ......}, the '0' or '1' indicate whether the corresponding label exists in the image.

why not just upload this file and save everybody some time... thanks...

I am very sorry for that, I didn't check the code when I first upload this repository, this is my fault, and now I am not in the lab. The class label is NOT important for segmentation task, you can feel free to remove the relevant part. And everyone want to use the class label please generate the multi-hot label by yourself, this is a multi-label classification problem.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[JaveyWang]

@JaveyWang name 'multi_hot' is not defined,what should I do.

function multihot() is a pseudo code, which means the results should like the forms of [0, 1, ...,0]. For example, if there are 20 classes and the image is belong to 1 and 3 class. The multi-hot label is looks like [0, 1, 0, 1, 0, ..., 0] (20 elements)。I am sorry for the ambiguity of my code, you can remove the part about the classification, it is not important.