Left hand vs. Right hand

[Xonxt]

Hi, maybe you could help me with something or have found a solution already.

First of all, I'm using a different approach for actually detecting the hands and hand-regions in the image (that works faster than the OpenPose one), and I'm also using Keras+TF, not Pytorch, but that is irrelevant.

I've noticed, that the hand-keypoint detector works much better (i.e. gives much more accurate keypoints) for one hand (the right one, I think), but works far worse for the other one. Example below:

[image]

I assume that you're supposed to know beforehand (pardon the pun), which hand you're processing, and then flip the image horizontally, when necessary? Or is there some other trick to it?

[Hzzone]

Firstly you should read a fragment of description from original paper: especially the following:

We directly use the body pose estimation models from [29] and [4], and use the wrist and elbow position to approximate the hand location, assuming the hand extends 0.15 times the length of the forearm in the same direction.

and

we flip the image left-to-right and treat it as a right hand.

So you know that there will be known that the detected hand bounding box is left or right because of doing body pose estimation. If you have trained your own hand detector, you should train it again to let it output the hand is left or right.

Secondly, you should check carefully that your network and weight are the same as original one, such as activation functions.

Hopefully it helps your work.

[Xonxt]

we flip the image left-to-right and treat it as a right hand.

So you know that there will be known that the detected hand bounding box is left or right because of doing body pose estimation. If you have trained your own hand detector, you should train it again to let it output the hand is left or right.

hi, yes, thank you, that's exactly it. I have already figured it out a few hours prior to your reply (literally as I was writing the original post) and it works fine with flipping the image. My hand tracker is a bit uncertain about which hand is right and which is left, if you cross your arms, i.e. move your LEFT arm to the right side, or vice-versa... but my application doesn't really foresee this situation, so it's fine.

Thank you anyway.

And yes, the model is correct and works perfectly well, it has been carefully and painstakingly converted ;-). The issue was only with the right vs. left confusion.