search

keel-210 / OpenPose-Rig

tf-openpose and unity IK
98 stars 28 forks source link

Hey, i've been working on a similar project #1

Closed ananttheant closed 6 years ago

ananttheant commented 6 years ago

Hey keel, I've been working on a similar project in which I have to use OpenPose Inside unity, could you guide me in doing so, or show me a path which would help me.

I don't have experience in Machine learning and TF, so it's really time taking for me. I saw your post on Qiita but due to the language barrier, I wasn't able to understand to full extent.

Hope for a quick response.

Regards, Anant

keel-210 commented 6 years ago

Hi Anant, I am not good at English,so please forgive obscure comment. First I execute tf-openpose, not OpenPose. So bone structure is slightly different. Second actually I don't execute tf-openpose inside unity. I execute tf-openpose on Anaconda virtual machine. And I get 3D pose data from variable named "pose_3d" in tf-openpose sample code run.py. Then I import 3D pose data to unity in the form of text data. So I may not be of help execution OpenPose inside Unity. I think that preparing python-executable environment in Unity maybe execute OpenPose.

Qiita supplement: Estimate_from_movie.py in my Qiita is the code to output 3D pose data in the form of .txt of each frame. IKSetting is the code to format 3D pose data and influence to IK. BoneController is the code to format 3D pose data and influence to Humanoid Bone.

Regards, KEEL

ananttheant commented 6 years ago

Ah, sorry! Actually, I'm trying to learn TF-OpenPose, how it works. In the end, I want to move a rigged character from the data I collect. So my question is more like "Is there any way to make a Rigged Character mimic the human on the webcam?"

I think Estimate_from_movie.py and BoneController will be what I need right?

And What version of Cuda did you use?

keel-210 commented 6 years ago

The Answer of your question is "Maybe yes.". But I think that Estimate_from_movie.py and BoneController should be modified about the form of input and output. Webcam image can get from cv2.VideoCapture(0). So input(variable "movie") of Estimate_from_movie.py should be modified. I guess that tf-openpose sample code run_webcam.py is helpful for that question. Similarly BoneController should be modified timing of calling PointUpdate() function. This code makes the head angle is too deep, so it need adjustment. And CUDA version is ...my PC have no GPU, so I use tf-openpose.