search

stevenjj / openpose_ros

A ros wrapper for the CMU openpose library
60 stars 25 forks source link

Skeleton_extract_3d_node + others #4

Closed MiguelARD closed 6 years ago

MiguelARD commented 7 years ago

Hi,

I am an undergraduate student at CFIS-UPC (Universitat Politècnica de Catalunya, also called Barcelona Tech, Spain). This summer I've developed a project during a summer internship at IRI (Institut de Robòtica i Informàtica industrial, also in Barcelona, Spain) in Perception and Manipulation Lab.

I have developed this project using as starting point your code (thank you again for your help). Additionally, I have implemented a ROS node (skeleton_extract_3d_node) to obtain 3d pose estimation from the initial 2d pose estimation using a depth image synchronized with the RGB image. I have worked with images that have been simultaneously captured with a Kinect Camera. To this aim, I have projected the 2d pose estimation body joint points onto the point-cloud of the depth image. It works quite well except in the case of some part of the body occlusion.

I am currently finishing a tech report where I explain all the changes I made and I will upload it soon. For the now, I have finished the code and a README. Of course, feel free to pick whatever you think convenient from the material I send.

CFIS-UPC: https://cfis.upc.edu/es Barcelona Tech: http://www.upc.edu/?set_language=en IRI: http://www.iri.upc.edu/

stevenjj commented 7 years ago

Congratulations on implementing the 3d skeletal extractor node!

I'm happy to see some of my comments turned into real code :) and it looks to be done well. 👍 . The documentation also looks pretty solid.

When I return to the lab, I will look at it more closely and enable the merge.

In terms of the body part occlusions, it's quite a common problem that can usually be resolved by feature and/or context-based reasoning. For some other practical applications, it may also be desirable to make a skeletal tracker based on kalman filters, particle filters, feature-based filters, etc.

While I do believe the original authors of openpose could have made a skeletal extractor (if they had the data), I believe the approach here can still be practical -- so well done!

MiguelARD commented 7 years ago

Thanks a lot! Without your help it wouldn't have been possible. I hope it is useful.