HowieMa / NSRMhand

[WACV 2020] "Nonparametric Structure Regularization Machine for 2D Hand Pose Estimation"
104 stars 16 forks source link

model structure in code incompatible with in paper #18

Open HollyDQWang opened 3 years ago

HollyDQWang commented 3 years ago

In paper the network layout was defined as using 3 structure stages and 3 keypoint stages, while in code the 6 stages were divided into 1 structure stage and 5 keypoint stages. What is the reason for such difference in terms of implementation or is it merely a typo?

HowieMa commented 3 years ago

It should be 3+3, as in the code image

HollyDQWang commented 3 years ago

The screenshot displays 1 + 5 if I understand it correctly, as it shows stage1 as using class Stage1 and stage2-stage6 as using class Stage though.

HowieMa commented 3 years ago

Sorry, but you may misunderstand the code. Please note that class Stage1 is just two 1x1 Conv layers, which follows OpenPose You can find that the output channels of stages 1, 2, and 3 are all self.lsh_c, which is the number of limbs. And the output channels of stages 4, 5, and 6 are "self.outc", which is the number of keypoints.

HollyDQWang commented 3 years ago

截屏2021-07-07 下午1 59 25

Please note that class Stage1 is just two 1x1 Conv layers, which follows OpenPose

I think I get what you mean, which is exactly why I was confused: the graph displayed in the paper said to use stage1 (with two 1x1 conv layers) three times for structure stage (limb stage as in your code), but in the code it was used only once. Am I missing something here?

HowieMa commented 3 years ago

Oops. It seems that our figure is incorrect. Thanks for pointing out this issue. the "Structure Stage 2" and "Structure Stage 3" should have the same structure as Statge t (t>1), and we should not use the blue background for it. And the Keypoint Stage 1, 2, and 3 also use the architecture of Stage t (t>1).

Actually, the main idea is that we still follow the architecture of CPM for hand pose.