How to use CIHP_PGN for parsing?

[hamaadtahiir]

Kindly share with me how CIHP_PGN was used and with what tensorflow version to obtain the human parsing

[nihirv]

The issues in the CIHP_PGN repo discuss this. I found this one particularly useful. https://github.com/Engineering-Course/CIHP_PGN/issues/38

[hamaadtahiir]

It does give me output but some labels are missing compared to the png files found in hr vton training data. I ran cihp pgn on the hr vton images to compare if I got the correct labels

[nihirv]

Not sure what you mean by labels here. The different segmentation classes? An example could be useful

[hamaadtahiir]

yes, by labels i mean different segmentations classes. For example:

0-Background 1-Hat 2-Hair 3-Glove 4-Sunglasses 5-Upper-clothes 6-Dress 7-Coat 8-Socks 9-Pants 10-tosor-skin 11-Scarf 12-Skirt 13-Face 14-Left-arm 15-Right-arm 16-Left-leg 17-Right-leg 18-Left-shoe 19-Right-shoe

[nihirv]

Ah yeah. In the cp_dataset files, they map the 20 classes to the 13 that they use:

        labels = {
            0: ['background', [0, 10]],
            1: ['hair', [1, 2]],
            2: ['face', [4, 13]],
            3: ['upper', [5, 6, 7]],
            4: ['bottom', [9, 12]],
            5: ['left_arm', [14]],
            6: ['right_arm', [15]],
            7: ['left_leg', [16]],
            8: ['right_leg', [17]],
            9: ['left_shoe', [18]],
            10: ['right_shoe', [19]],
            11: ['socks', [8]],
            12: ['noise', [3, 11]]
        }

[hamaadtahiir]

yes, but when I run cihp_pgn on hr vton data, it fails to give segmentation labels for the lower part of body. I am using tensorflow version 2.8.0 with gtx 3090

[nihirv]

Interesting. Maybe best to wait for the authors response then. I've run it on additional data and fed that into the model just fine (though training quality seems significantly worse when I add more data to the dataset... maybe this has something to do with it). Also, I think I had to use tensorflow 1.15 to be able to run the model - it wasn't written for tensorflow 2

[hamaadtahiir]

hr vton model is just bad in generalization. I used the simple human parser as well, changed the labels and still results were very poor

[RAJA-PARIKSHAT]

@nihirv I am able to get the parsed image from CIHP_PGN, but the format is not ok, somehow. I am getting an error because of the image format, when I load the image given by the authors, it is loaded as mode= "P" in the PIL image, whereas my parse image is loaded as mode = "RGB" image, what is going wrong there?

I managed to get this image

I am getting this error

[nihirv]

@RAJA-PARIKSHAT "P" means that it is a palletised image. You need to write some code which will take the RGB image created by CIHP_PGN and map it to the palette that is used by HR-VITON. You can open one of the parse images they provide in their dataset (in PIL "P" mode) to see what a palette image looks like - and you can look at the cp_dataset files to see the IDs they use for each of the classifications made by CIHP_PGN.

https://stackoverflow.com/questions/67967366/how-to-write-image-with-palette-information might help you get started

[Atotem]

Hi guys, sorry for my late response. I couldn't avoid reading your thread. I have been working in this for a month already. My conclusions, don't try to test the model with data that the model was not trained with, specifically human models. Apparently, the model is pixel sensible just for the case of the 'humans dataset' just try changing one pixel from the original dataset for testing and you will see. Anyway, I have a code to parse the human figures with CIHP model. However, you need dedicated GPU, otherwise it will crash with high resolution pictures. Also, I tried downsizing the resolution to reduce the time spent per picture. For my 300 images dataset, it took like 6 hours. I also recommend downsizing because the model is more accurate when parsing the body parts. The resize the outcome to the original dimensions. I will anyway trained the model and see what happens with external data. I will publish my code to parse with CIHP later.

[Atotem]

@RAJA-PARIKSHAT Also, to get the PIL format, you will need to do this:

from PIL import Image
# assuming you read the image with opencv
im = Image.fromarray(im).convert('L')

[koo616]

Hello guys. I just created 'Preprocess.md', the description for our preprocess including human parsing. Please check it :)

[yarfomin]

Hi guys, sorry for my late response. I couldn't avoid reading your thread. I have been working in this for a month already. My conclusions, don't try to test the model with data that the model was not trained with, specifically human models. Apparently, the model is pixel sensible just for the case of the 'humans dataset' just try changing one pixel from the original dataset for testing and you will see. Anyway, I have a code to parse the human figures with CIHP model. However, you need dedicated GPU, otherwise it will crash with high resolution pictures. Also, I tried downsizing the resolution to reduce the time spent per picture. For my 300 images dataset, it took like 6 hours. I also recommend downsizing because the model is more accurate when parsing the body parts. The resize the outcome to the original dimensions. I will anyway trained the model and see what happens with external data. I will publish my code to parse with CIHP later.

Hello! Thank you for the information you provided. Could you please tell me about your results? Did you train the model on your data, which results did you have if so?

[Atotem]

@yarfomin Yes, at first I was able to train and render some results. However apparently I have some issues with the parse map because I'm not having the desired agnostic representation images with your code. I'm working in a Google Colab notebook developing the whole pre-process pipeline, you just need the images with the human models. However, Google updated Colab 6 days ago and now I can't import tensorflow 1.15. I'm trying to figure out another way now. But almost there.

[yarfomin]

@yarfomin Yes, at first I was able to train and render some results. However apparently I have some issues with the parse map because I'm not having the desired agnostic representation images with your code. I'm working in a Google Colab notebook developing the whole pre-process pipeline, you just need the images with the human models. However, Google updated Colab 6 days ago and now I can't import tensorflow 1.15. I'm trying to figure out another way now. But almost there.

@Atotem I had the same issue in the Colab, you can use sudo update-alternatives --config python3 and choose python 3.6

[khurramkhalil]

Hi guys, sorry for my late response. I couldn't avoid reading your thread. I have been working in this for a month already. My conclusions, don't try to test the model with data that the model was not trained with, specifically human models. Apparently, the model is pixel sensible just for the case of the 'humans dataset' just try changing one pixel from the original dataset for testing and you will see. Anyway, I have a code to parse the human figures with CIHP model. However, you need dedicated GPU, otherwise it will crash with high resolution pictures. Also, I tried downsizing the resolution to reduce the time spent per picture. For my 300 images dataset, it took like 6 hours. I also recommend downsizing because the model is more accurate when parsing the body parts. The resize the outcome to the original dimensions. I will anyway trained the model and see what happens with external data. I will publish my code to parse with CIHP later.

Dear @Atotem , Just wanting to know if you are planning to share your code? I have my implemented version as well, we can reconcile with each other code for better outcome may be. Thanks