How can we use hand detector from OpenPose

[seva100]

@michalfaber, thank you for your awesome work. I am preparing a project that relies on pose estimation and requires hand landmarks. I was wondering if hand detector from OpenPose can also be used within your Keras version. Are you going to adapt it? May be, it is easy for me to adapt it myself?

[michalfaber]

Thanks @seva100 I’ve had a look at the hand detector from OpenPose. It uses different architecture - more vgg layers but fewer layers in total. Writing keras model based on proto file is the easiest part of the task. Probably more difficult would be to understand how actually this model was trained. Sure you can try to adapt it. Right now I focus on the training part in this keras version of the project.

[seva100]

@michalfaber thank you, will keep you informed about any progress.

[Zumbalamambo]

@michalfaber how do I draw heatmap around the entire body only?

[michalfaber]

@Zumbalamambo You can use segmentation data for the entire body. In this project we use only keypoints.

[Zumbalamambo]

can you please tell me how can I use the segmentation data or can you point me in a direction where I can do segmentation?

[michalfaber]

@Zumbalamambo Here is an example how you can draw segmentation using coco api (downloadable from https://github.com/cocodataset/cocoapi). Have a look at the source code for coco.showAnns

%matplotlib inline
from pycocotools.coco import *
import cv2

coco = COCO("./dataset/annotations/person_keypoints_val2017.json")
ids = list(coco.imgs.keys())
img_id = ids[322]
ann_ids = coco.getAnnIds(imgIds=img_id)
img_anns = coco.loadAnns(ann_ids)

img = cv2.imread("./dataset/val2017/%012d.jpg" % img_id ) 
plt.imshow(img[:,:,[2,1,0]])
coco.showAnns([img_anns[0]])

[Zumbalamambo]

thank you ... May I know the dataset and files that I need to download from this so as to test it?

[michalfaber]

@Zumbalamambo Yes

[oakkas]

Any update on hand landmark estimation with keras?

[JasOlean]

I also would like to know about hand pose estimation with keras.