Pose model trained from scratch with COCO data does not work

[shunsuke227ono]

Hi @leetenki , really thanks for your awesome implementation!! I would like you to help me if possible. I'm running your training script to learn pose model from scratch with COCO data, but the trained model I get does not work. Here are what I did;

1. Setup training data following with README(https://github.com/DeNA/Chainer_Realtime_Multi-Person_Pose_Estimation#train-your-model).
2. Checked data generator (https://github.com/DeNA/Chainer_Realtime_Multi-Person_Pose_Estimation#check-data-generator) and pafs, heatmaps seemed okay.
3. Training procedure is like below. (still training now)

78          505140      0.0345071
     total [##########################################........] 84.19%
this epoch [#######################################...........] 78.66%
    505140 iter, 78 epoch / 600000 iterations

4. Tested with the saved models during training like below

python3 pose_detector.py posenet result/20171206/model_iter_410000 --img data/person.png

=> But it doesn't detect anything in data/person.png. (like, pose_detector(img) returns empty array here: https://github.com/DeNA/Chainer_Realtime_Multi-Person_Pose_Estimation/blob/master/pose_detector.py#L507)

Do you have any ideas or suggestions about this problem? Thank you.

[shunsuke227ono]

2. Checked data generator (https://github.com/DeNA/Chainer_Realtime_Multi-Person_Pose_Estimation#check-data-generator) and pafs, heatmaps seemed okay.

For this part, I got result like this

I guess pafs and heatmaps work okay, but can you tell if mask works okay?(I cannot see mask on these images)

[leetenki]

Yes... there was some bugs in the Mask generator in the first version of source code. But we fixed it afterwards. If you pull the latest version code and check the mask again, it will work fine.

[leetenki]

I'm really sorry...

[leetenki]

We found that bug a week after we released the source code...

[shunsuke227ono]

@leetenki Thanks a lot for your quick reply :) I'll try to train it with the latest code again.

[shunsuke227ono]

Hi, it seems my training problem is not because of the bug you mentioned. My code for generating mask was actually newest, and saw some images like below with mask too.

But sill, the trained model does not work as I mentioned here 🤔 (https://github.com/DeNA/Chainer_Realtime_Multi-Person_Pose_Estimation/issues/20#issue-281347353) Would be helpful if you have any suggestion. Thanks!

[leetenki]

In the first version of our implementation, the trainer could not evaluate mask loss correctly, and it couldn't detect anything after more than 40000 training iterations same as you say. So I think it will work fine if you train the model again using the latest version of our code.

[shunsuke227ono]

@leetenki

Thanks for the quick response. I am training with the latest code again, but trained model does not detect any person. (I'm using a model gotten from 40000 training iterations whose loss is 6 40000 0.0420941 0.0475392 in the log. But pose_detector(img) returns empty array here: https://github.com/DeNA/Chainer_Realtime_Multi-Person_Pose_Estimation/blob/master/pose_detector.py#L507) Do you think it's because the training code still does not evaluate mask loss correctly?

---

⇩ Im running the latest version of code. here are git-logs.

ono@xxxx:~/Chainer_Realtime_Multi-Person_Pose_Estimation⟫ git log train_coco_pose_estimation.py
commit 1fda1d600b579f4a341f287d8354a71729e4cced
Author: Naoki Kato <naoki.kato@o-09119-mac.local>
Date:   Tue Dec 12 11:06:43 2017 +0900

    fix MultiprocessIterator's bug

commit 8db8f743e0206bfa627d281d2a13e36fe2a9ad96
Author: Naoki Kato <naoki.kato@o-09119-mac.local>
Date:   Tue Nov 28 15:17:52 2017 +0900

    update train_coco_pose_estimation.py

commit e0c6d3869ccd18c5f9cbae46286ec6d2b7b81d1b
Author: Tianqi Li <tianqi.li@o-08198-mac.local>
Date:   Wed Nov 15 18:24:11 2017 +0900

    first commit

ono@xxxx:~/Chainer_Realtime_Multi-Person_Pose_Estimation⟫ git log coco_data_loader.py
commit 4fd6b5340ed0615597b68770babea0604c11614c
Author: Naoki Kato <naoki.kato@o-09119-mac.local>
Date:   Tue Nov 28 15:16:21 2017 +0900

    fix background channel of heatmap

commit 9274755e2a58b3fda8c8447955eb954f8c7bb2a6
Author: Tianqi Li <tianqi.li@o-08198-mac.local>
Date:   Thu Nov 16 18:36:49 2017 +0900

    fix mask bug

commit e0c6d3869ccd18c5f9cbae46286ec6d2b7b81d1b
Author: Tianqi Li <tianqi.li@o-08198-mac.local>
Date:   Wed Nov 15 18:24:11 2017 +0900

    first commit

[leetenki]

Em... can you send us your trained model and the image you use to detect person? This is my email address. tianqi.li@dena.com.

[leetenki]

This problem was solved.