search

zh-plus / video-to-pose3D

Convert video to 3D pose in one-key.
MIT License
646 stars 130 forks source link

Adding Posenet or HrNet as 2d Detectors #1

Closed basicvisual closed 5 years ago

basicvisual commented 5 years ago

Hi , could you also see the possibility to add 2d detectors such as

posenet and simpleJHrNet

I was also looking into lifring it from the following repositoy but i had not managed to get good resutls

3d pose this takes open pose as input but i think it could be extended to other detectors as well . What are your thoughts ?

zh-plus commented 5 years ago

@basicvisual Sorry for the delayed response (I refactor the original project inspired by the last repo I mention in the Acknowledgment).

Two weeks ago, I've tried the HR-Net as 2d joints detectors, but it seems that the 2d result is much unstable like this: hr_pose_actor2

compared to the result of AlphaPose: alpha_pose_actor2

I'll try to find if there are still any bugs in the processing of HR-Net causing this instability in my spare time. If the bug is found and fixed, I'll @ you here.

As for using the posenet and openpose as 2d pose detector, I would not test them for lower accuracy than HR-Net (SOTA in 2d joint detector) and AlphaPose. But maybe you can make use of this repo by adding them in the joints_detectors folder and test by yourself. I'll sure that the structure of this repo is great for extensibility.

Feel free to make contact with me if there is any bugs of my code or enhancement advises of you. (I'll pay attention to issues this time...... I'm really sorry for such delayed response.)

basicvisual commented 5 years ago

Thanks for the explanation. I think it could also be the pre trained model. According to what i understand the pretrained weight pose_hrnet_w48_384x288.pth should be the one with best results for single user. I will look into it . Thanks again

sebo361 commented 4 years ago

Hi @zh-plus & @basicvisual, i am looking for a configuration to get the most accurate single person 2D keypoint detection available (no matter how long inference take). I looked into Alphapose repo and found a --flip flag which might improve accuracy but its not used in this repo right? Is the duc_se.pth model with ResNet152 backbone (see table)?

Any other ideas to improve the accuracy of 2D keypoint detection?