search

fabro66 / GAST-Net-3DPoseEstimation

A Graph Attention Spatio-temporal Convolutional Networks for 3D Human Pose Estimation in Video (GAST-Net)
MIT License
313 stars 70 forks source link

Recommended way to get 2D keypoints #3

Closed vsantosu closed 4 years ago

vsantosu commented 4 years ago

Hello there.

I'm trying to feed some Open Pose COCO keypoints in the same format as the example in the repo. In fact, I'm extracting the COCO 2D points of the baseball video using Open Pose but I get some occluded parts that shows in the x,y coordinates as:

-1, -1

This causes GAST-NET to go crazy when trying to reconstruct the 3D points. Can you provide some insight on the best way to prevent this? Which Model did you used to get the keypoints in the example data?(baseball)

fabro66 commented 4 years ago

Hi, there. In our repo, we first use yolo3 and sort to detect and track people. Then we lift 2D poses predicted from HRNet to 3D poses. In the future, I will release the code for how to track and estimate multi-person 3D poses from the video in real-time.

vsantosu commented 4 years ago

That sounds like an interesting approach, @fabro66 ...

So you combine object detection and a keypoint extraction model. Is this repository you are talking about publicly available?

I see that you have an OpenPose PyTorch in your repository list, but I cannot find a repo with HRNet+Yolo.

I would like to give it a shot to extract the keypoints of some videos I have.

Thanks again for your help, this is high-quality work!

fabro66 commented 4 years ago

Hi~ Thank you for your interest in our work. Recently, I am arranging the whole project of YOLOv3+SORT+HRNet+GAST-Net. Sorry that this project has not been uploaded to my GitHub yet. I will upload it to this repo after finishing it.

vsantosu commented 4 years ago

Thanks so much @fabro66 , if there is something I can help you with, please let me know.

Thanks for your amazing work!

fabro66 commented 4 years ago

Thanks so much @fabro66 , if there is something I can help you with, please let me know.

Thanks for your amazing work!

Ok. Thanks. If you have any other questions, please feel free to contact me.