search

Fang-Haoshu / multi-human-pose

Training code for the SPPE and SSTN of our RMPE framework
30 stars 11 forks source link

Is the implementation in caffe coming soon? #2

Open lxltaibo opened 7 years ago

lxltaibo commented 7 years ago

hi, we want to used the code in caffe, are you still working on caffe implementation? Is it coming soon?

Fang-Haoshu commented 7 years ago

Hi,

I have a script to generate stacked hourglass in caffe, do you need it?

lxltaibo commented 7 years ago

Yes, thank you.

Fang-Haoshu commented 7 years ago

Hi

Here is the script:https://drive.google.com/open?id=0BxORzoJl8N9PS0lHbFJHLWQ2RVJ1blRtYmI4b0ZpUVJMRWRZ. The caffe version is here: https://github.com/Fang-Haoshu/RMPE

Place it under examples/rmpe/ and run the script from $RMPE_root

Currently the input format is bbox format, I also implement the centerscale version as original stacked hourglass but not tested. You can modify the param in the script.

However, the reason I do not public it now is that, it's hard to train using Caffe, always loss=nan, I only success in training it for 1 time and can not reproduce it. If you find the reason,please let me know.

Hope this may help

Best

lxltaibo commented 7 years ago

Hi, I use you script to generate the training prototxt, and I have some question when training.

  1. The prototxt generated by the script is not the same as the Newell paper, e.g. in the papaer, elementwise adding is used (three input) between hourglasses, and in your implementation, the concat (two input) and then the elementwise adding(two input) is applied, why? (By the way, I try to generate the prototxt the same as the paper, but failed when training.)

  2. I also find the loss=nan, but just in specific GPU in my PC. In detail, GPU 0, loss is ok , but when using GPU 1 , the loss is nan. The results can be reproduced.

Fang-Haoshu commented 7 years ago

Hi

  1. In fact there are two version of Stacked hourglass. My implementation is V1 and later Newell update the second version. The performance of V1 is 89.7mAP on MPII. If possible, can you share the prototxt of the new version? I can have a look to see what's the problem.

  2. Yes, that's really weird. I once guessed it's a bug in CuDNN or even GPU device.

lxltaibo commented 7 years ago

Thanks. The train prototxt with two stacked hourglass modules of V2 is shown below, the main difference is:

  1. There is only one residual module after pooling in the hourglass module;
  2. The processing between hourglass is different;

train.txt

Fang-Haoshu commented 7 years ago

As far as I can see, there is no problem with the network structure... Maybe you can try to init all the conv with xavier

lxltaibo commented 7 years ago

Thanks for reply. I change the version of the BN layer, and the loss is no longer nan. Maybe there is something wrong with the BatchNorm layer.

The hourglass prototxt of V2 can also obtain the right key point heatmap.

Fang-Haoshu commented 7 years ago

Oh yes, the official BN layer is not that perfect:)

anshan-XR-ROB commented 7 years ago

Hi, Fang, I have try your caffe code about SHG. The pose estimation result is poor. The accuracy_heatmap = 0.225325 after 70000 iters on MPII dataset. What should I do to get 89.7mAP on MPII?

Fang-Haoshu commented 7 years ago

Hi Anshan,

Did you try the prototxt attached by lxltaibo in this issue and also change the BN version?

anshan-XR-ROB commented 7 years ago

Thank you for Fang. I will try @lxltaibo 's prototxt.

anshan-XR-ROB commented 7 years ago

Hi, Fang, I have another problem for torch. table.insert(out, tmpOut) I want to know the meaning of this line. I only use tmpOut4 for output and the result is poor. Shoud l concat all tmpOut to output?

Fang-Haoshu commented 7 years ago

@AnshanTJU For the caffe prototxt, that's a 2 stack version. You can try to extend it to the 8 stack version

Fang-Haoshu commented 7 years ago

@AnshanTJU For torch, yes, it's for 8 stacked hourglass

wutao20171222 commented 6 years ago

I try @lxltaibo 's prototxt.But I can't run successfully。Tip: The roi area of opencv is out of bounds why？

soulslicer commented 6 years ago

@lxltaibo

Do you have the weights and solver for your model? I wish to try it but don't want to train from scratch

sixsixsix666 commented 5 years ago

Thanks. The train prototxt with two stacked hourglass modules of V2 is shown below, the main difference is:

  1. There is only one residual module after pooling in the hourglass module;
  2. The processing between hourglass is different;

train.txt

@lxltaibo Thanks for sharing your training prototxt file! I found a "EltwiseAffine" layer followed by "BatchNorm" layer in your prototxt. Can you tell me what this layer is for? :)