Is there any mask for some people?

[LEESEYUN]

Thank you for reading my question!

I know that you have to remove some annotation of people who has little annotation(<5), who has little scale(<32*32) and who is so close to 'main_person'. Is there any other mask_miss to image level?? I counldn't find that part.. Thank you!!

I have another question.

Is Black mask_miss? (0,0,0) Or white mask_miss? (255,255,255)

[white mask_miss]

[black_miss]

[anatolix]

Mask miss is multipled with the loss, so masked areas are 0. I.e. second mask is real mask miss. First mask is just for visuals.

Reason to apply mask miss is if p["num_keypoints"] <= 0 i.e. person is segmented but have no keypoints(joints)

"people who has little annotation(<5), who has little scale(<32*32) and who is so close to 'main_person'" are not masked, they just can't be selected as main person of image. This repo contains a bug, this people are neither masked nor joints passed to model, i.e. this repo couldn't be trained correctly. Fix available in my fork of this projects

[michalfaber]

@LEESEYUN @anatolix I am in the process of overhauling the repo. Indeed there is a bug but it is not critical.

[anatolix]

Actually I were unable to reproduce good quality for ski.png not with C++ nor with Python augmentation, I always get some additional points for parts. See this https://github.com/michalfaber/keras_Realtime_Multi-Person_Pose_Estimation/issues/39 Learning rate doesn't helps.

After I've fixed generate hdf5 it was immediately fixed see this picture: https://github.com/michalfaber/keras_Realtime_Multi-Person_Pose_Estimation/issues/41

So I think it is critical and it is the reason of bad quality.

[kevinlin311tw]

I am wondering that did you conduct performance evaluation on the public datasets such as COCO keypoint dataset? I feel this could be the a best way to validate the correctness of the codes.

[anatolix]

@dremovd made COCO test on COCO validation dataset, see here: https://github.com/anatolix/keras_Realtime_Multi-Person_Pose_Estimation/blob/master/coco_metric.py

But this metric is not really informative since we TRAIN on validation dataset as well. We use COCO validation dataset for training except first 2650 images, so it is not informative.

Approx 0.58 for C++ augmentation, 0.61 for python augmentation and 0.63 for caffe converted model, but I've not sure caffe model haven't seen different images compared to our model.

Probably we could make submission to test on COCO hidden dataset. But I haven't done it.

p.s. And btw original work trained on COCO 2014 not COCO 2017, they have near 100% intersecton in images, but markup could be different