search

Nor-s / mediapipe-to-mixamo

mediapipe landmark to mixamo skeleton
GNU General Public License v3.0
33 stars 7 forks source link

How to run the repo? #3

Closed timegoby closed 1 year ago

timegoby commented 1 year ago

I wonder how to get the result windows shown in the repo. And why there is no load model(the model download from mixamo is .fbx) code?

Nor-s commented 1 year ago

Sorry for the poor documentation. Currently, these sources are included in the Anim project, receiving model data from the Anim program and processing it in Python code.

.fbx -> C++ assimp -> .json -> python code

timegoby commented 1 year ago

Got it, thanks a lot !

timegoby commented 1 year ago

I have get the data of pose position from mediapipe. And I want to change it to quaternion data used in .fbx model from mixamo, just like "kalidokit" which can change it to quaternion data used in .vrm model. Is this repo can help? For I do not have any knowledge about the 3d model render.

Nor-s commented 1 year ago

This repo I developed was initially made for school homework, so it is probably difficult for others to see. So I don't think it will help.

You'd probably better take a look at this project. https://github.com/cgtinker/BlendArMocap

As a side note, the way I did it only requires knowledge of the model's skeleton hierarchy. I first made a tree of mediapipe points. (to have coordinates relative to the parent) You can get the quaternion by taking the position hip as (0, 0, 0) and applying IK to the child bone based on that.

And the 3D skeleton knowledge needed to write them can be acquired in this tutorial. https://ogldev.org/www/tutorial38/tutorial38.html

timegoby commented 1 year ago

Thanks a lot, I saw the repo given by you, that's really help! I find Anim project render the .fbx model. I just want to transfer the position data of bones from mediapipe to rotation data which used in .fbx model. Is the transfer code in your project? Or used other tools to achieve that? Thx again.

Nor-s commented 1 year ago

There is no code to send. I just chose the simple way. Just save it as a custom json file and load the json file in python code. If you want to transfer directly, there is a way to use redis. (or ipc, shared memory)

timegoby commented 1 year ago

Thanks, got it, but I may not have described my question clearly. What I want to know is that the data from mediapipe is 3d position of bones, but the the data using in .FBX model is the rotation data of bones. And I wonder the transfer code between the two type of data, for you project seems achieved that.

Nor-s commented 1 year ago

calc_animation this code change 3d positions to rotation.

timegoby commented 1 year ago

Got it, thanks a lot.