Masking the Input Image

[WillBrennan]

https://github.com/michalfaber/keras_Realtime_Multi-Person_Pose_Estimation/blob/ff482e118ace9c9e6e3d0291fb4f2cca385d0ae7/training/dataset.py#L129

Hi,

Great repo! But is this correct? you're applying the mask to the input image, when the paper says to only apply it to the PAF and heatmap outputs during training.

I couldn't find a similar snippet in https://github.com/ZheC/Realtime_Multi-Person_Pose_Estimation

If this is intentional, why is it needed and what effect did it have?

[michalfaber]

I applied the masks to the input images as part of an experiment. The idea was that if I apply the mask before training, I would not need to apply the mask to multiple outputs - I expected better speed of training. Right now I am not so sure if that helps. You can comment out the line https://github.com/michalfaber/keras_Realtime_Multi-Person_Pose_Estimation/blob/ff482e118ace9c9e6e3d0291fb4f2cca385d0ae7/training/dataset.py#L197

[WillCheung2016]

@michalfaber Thanks for your excellent work. Actually I think masking unlabelled part of the image before it is fed into the model for training is a good idea. By doing so we don't need to apply masks on the outputs. I may be wrong, but in my opinion, the value used for masking on the input image needs to be 128, not 0.

[rafikg]

@michalfaber By applying the mask to the image before fed it into the model is a good idea and we don't need to apply masks on the outputs. In addition, in this case we don't need build two graphs one for training and the other for testing.