Conversion to FBX

[carlosedubarreto]

I saw the great work you all done to have the bvh conversion working.

here is my approach to convert to FBX adapting the code from VIBE mocap:

install pandas pip install pandas

install bpy via pip pip install bpy

if pip install bpy doesnt work, install using conda (if you have conda installed) conda install -c kitsune.one python-blender

copy blender 2.82 files to the python executable. If you use conda on the base enviroment or virtual enviroment, you can fid tha directory using

conda env list

Copy the 2.82 folder from blender download nd paste to the enviroment you are using.

and save the "fbx_output_EasyMocap" at this folder:"scripts/postprocess"

and finish doing these instructions of downloading the SMPL for unity https://github.com/mkocabas/VIBE#fbx-and-gltf-output-new-feature

then you can test using the demo video file

python code/demo_mv1pmf_smpl.py demo_test --out out_demo_test --end 300 --vis_smpl --undis --sub_vis 1 7 13 19 --gender male

then test the fbx conversion with python scripts/postprocess/fbx_output_EasyMocap.py --input out_demo_test/smpl/ --output out_demo_test/easymocap.fbx --fps_source 30 --fps_target 30 --gender male

The fbx conversion script: fbx_output_EasyMocap.zip

[carlosedubarreto]

Just added this if people are having problems with the BVH conversion provided, or want another solution to convert. Thats not really an issue.

[chingswy]

Thank you Carlos! I have noticed the code in VIBE, the reason why I use bvh is that I can retarget the motion to other avatars. I will check this code and add it into convert2bvh.py.

[carlosedubarreto]

I understand your point. I was trying some different methods that could make a better output. now I'll start tweaking your bvh code and vibes to see if I can have a more fluid animation (and one that the foot doesnt go crazy and that can follow the hips movement)

BTW, putting the animations side by side, your bvh solution bring much more quality to the exported result.

[carlosedubarreto]

@chingswy Just to share. I finish a beta version of a blender addon that does this import using another method. I plan to add the hability to import also frankmocap and vibe.

If someone is interested, here is the file and link

Mocap_PE_data_import_0.1.zip

https://gum.co/mocap_import the link is interesting so It will be updated everytime I finish a new version.

[chingswy]

Wonderful work! It seems that it is easy to add hand/face skeletons to Armature?

[carlosedubarreto]

Thanks a lot @chingswy Hand/Face are going to be the 3º phase of development. I added support to Frankmocap and Vibe. 2º Phase is the option to add MediaPipe to be used right inside of blender (to the ones that want to build a mocap data more easily)

then I plan to add face and hands...

[carlosedubarreto]

@chingswy Just added and tweaked your code to import the smpl json inside of blender, in the addon I'm developping. Now it can be more easy to people to get the 3d animations.

[chingswy]

Thank you @carlosedubarreto ! Where can I find the tutorials? I wonder if it is OK to get the animations with only the skeleton results and how you retarget the different skeletons. I will add your tutorial to the Readme page when evereything is ready. Thanks for your contribution!

[carlosedubarreto]

The addon was done in a way to be used without the need of a tutorial (well I tried to build in a way to be used without tutorial, but I may be wrong 😅)

There is this post on twitter that shows how can someone get the animations using the addon

https://twitter.com/carlosedubarret/status/1386979472746946560

That is basically, locate where the SMPL for Maya FBX is located, and then choose where the SMPL json files are located.

I plan to make tutorials on installing easymocap on windows (as I did for VIBE and Frankmocap). To be completely honest, I plan to make a series of tutorials, becaus, the installation of easymocap in windows is very simples, but the calibration is where I think the tutorial will make a huge difference.

And I learned a lot of thing from trying to calibrate since version 0.1 some months ago.

I'm trying to build theses installation tutorials during the week, but it is more complicated. More complicated to do narrated tutorials, but I'll try at least to creat some tutorials with text only.

Well, I'll keep you updated about it, so many more peopple can use your amazing tool (seriously, I think it is amazing, thats why I'm devoting so many time with it).

and thanks for it and for all the attention you´ve geve me to help on making things work.

[kenchikuliu]

I can retarget the motion to other avatars.

Could u tell me how to retarget the motion to other avatars? and How I can rig my own smpl model?

[carlosedubarreto]

@kenchikuliu you can use Rokoko Studio, its a free addon and very easy to use.

https://github.com/Rokoko/rokoko-studio-live-blender

or here to get the latest release at the moment 1.2.1 https://github.com/Rokoko/rokoko-studio-live-blender/releases/download/v1-2-1/rokoko-studio-live-blender.zip

this tutorial could help you on that. I thank it was the one I saw,and didnt have problems when I did retargeting https://www.youtube.com/watch?v=iKkz3e1dB7Q

If you can, could you post a result so we can see the retargetting? I would love to see it.

[kenchikuliu]

@kenchikuliu you can use Rokoko Studio, its a free addon and very easy to use.

https://github.com/Rokoko/rokoko-studio-live-blender

or here to get the latest release at the moment 1.2.1 https://github.com/Rokoko/rokoko-studio-live-blender/releases/download/v1-2-1/rokoko-studio-live-blender.zip

this tutorial could help you on that. I thank it was the one I saw,and didnt have problems when I did retargeting https://www.youtube.com/watch?v=iKkz3e1dB7Q

If you can, could you post a result so we can see the retargetting? I would love to see it.

Thx a lot, by the way, could u tell me any other method to get a rigged smpl model from my own smpl model?

[carlosedubarreto]

@kenchikuliu You could try mixamo to rig your character or autorig pro (paid)

If you already have a SMPL model, I think you could choose yur model when using the mocap_import addon. It enables you to use other smpl fbx. Probably that can be what you need

[carlosedubarreto]

Thank you @carlosedubarreto ! Where can I find the tutorials? I wonder if it is OK to get the animations with only the skeleton results and how you retarget the different skeletons. I will add your tutorial to the Readme page when evereything is ready. Thanks for your contribution!

@chingswy I just created 2 tutorials, one for instalation and the other with tips that I've learned through the time I'm testing easymocap

the instalation video there is 10 minutes that I'm totally lost trying to configure the smpl files LOL... but I couldnt make it better because of the time available I had. It took me more than 6 hour to do them 😅

easymocap instalation on windows https://youtu.be/8gGua4xek90

Easymocap Calibration https://youtu.be/4qU4wZ3EU64

[kenchikuliu]

@kenchikuliu You could try mixamo to rig your character or autorig pro (paid)

If you already have a SMPL model, I think you could choose yur model when using the mocap_import addon. It enables you to use other smpl fbx. Probably that can be what you need I mean my smpl model is just a mesh(obj file, only mesh and texture without skeleton), how I can get a rigged smpl model(fbx) from my own smpl model(obj file) to replace basicModel_m_lbs_10_207_0_v1.0.2.fbx?

[carlosedubarreto]

I mean my smpl model is just a mesh(obj file, only mesh and texture without skeleton), how I can get a rigged smpl model(fbx) from my own smpl model(obj file) to replace basicModel_m_lbs_10_207_0_v1.0.2.fbx?

That I dont know for sure, but maybe if you copy the armature from basicModel_m_lbs_10_207_0_v1.0.2.fbx, and put it to your model, maybe it can work. (propably you will also have to transfer the weights from the fbx file to your model, or else only the bones will move and your mesh will not.

[tianyu06030020]

I saw the great work you all done to have the bvh conversion working.

here is my approach to convert to FBX adapting the code from VIBE mocap:

install pandas pip install pandas

install bpy via pip pip install bpy

if pip install bpy doesnt work, install using conda (if you have conda installed) conda install -c kitsune.one python-blender

copy blender 2.82 files to the python executable. If you use conda on the base enviroment or virtual enviroment, you can fid tha directory using

conda env list

Copy the 2.82 folder from blender download nd paste to the enviroment you are using.

and save the "fbx_output_EasyMocap" at this folder:"scripts/postprocess"

and finish doing these instructions of downloading the SMPL for unity https://github.com/mkocabas/VIBE#fbx-and-gltf-output-new-feature

then you can test using the demo video file

python code/demo_mv1pmf_smpl.py demo_test --out out_demo_test --end 300 --vis_smpl --undis --sub_vis 1 7 13 19 --gender male

then test the fbx conversion with python scripts/postprocess/fbx_output_EasyMocap.py --input out_demo_test/smpl/ --output out_demo_test/easymocap.fbx --fps_source 30 --fps_target 30 --gender male

The fbx conversion script: fbx_output_EasyMocap.zip

hello @carlosedubarreto, I use your supplied fbx_output_EasyMocap.py and successed output the fbx file, But in blender it looks like static with no animation, just like this:

[carlosedubarreto]

Hello @tianyu06030020 , there is a long time that I do not play with that, but I would suggest you to take a look at this addon I made sometime ago.

https://carlosedubarreto.gumroad.com/l/mocap_import

This addon enables you to import from several mocap solution, including easymocap.

And if you want a full solution, you can try CEB Easymocap. that you can get for free (using the code on the description) https://carlosedubarreto.gumroad.com/l/ceb_easymocap_blender

[tianyu06030020]

@carlosedubarreto , so appreciated, I recheck the json path, that works already.

[carlosedubarreto]

great, thats even better. I wish you a nice mocaping 😃

[xuguozhi]

your bvh solution bring much more quality to the exported result.

BVH quality is better than FBX? I am a little confused.

[carlosedubarreto]

If I remember correctly both method goes through blender, the difference would be the process done inside of blender and not the output itself. I probably referred as BVH and FBX to tell which method I used.

[xuguozhi]

If I remember correctly both method goes through blender, the difference would be the process done inside of blender and not the output itself. I probably referred as BVH and FBX to tell which method I used.

@carlosedubarreto another problem. When I imported the fbx file in Blender, strange results appear like image below however, the demo result seems normal:

[carlosedubarreto]

yeah, its normal. To solve this what I do is get the root bone, remove all the keyframes, and rotate it 90 degrees on the y axis. remove all the frames only from the root bone.

[xuguozhi]

another question, the exported fbx frame rate is locked to 24 fps, why?

yeah, its normal. To solve this what I do is get the root bone, remove all the keyframes, and rotate it 90 degrees on the y axis. remove all the frames only from the root bone.

[carlosedubarreto]

I dont know why. Maybe to get the script ready faster, instead of detecting the fps, a fixed fps was chosen, or maybe it was not possible to detect, since the the conversion to fbx happen over the triagulated data, and not the video.

those are just guesses, I cant tell for sure.

[0Xiao0]

yeah, its normal. To solve this what I do is get the root bone, remove all the keyframes, and rotate it 90 degrees on the y axis. remove all the frames only from the root bone.

@carlosedubarreto hello, I also have this problem like that, could you tell me the details of how you solve this problem? I can't understand what you means to remove all the keyframes,and rotate it 90 degrees on the y axis. It means that I couldn't have the motion pose? I have no idea.

[carlosedubarreto]

Hello @0Xiao0 , I mean in blender.

If you use the script to import the easymocap data in blender, by default, the root bone (the parent of all bones) will be keyframed.

The easiet way that I found to explain to rotate the character is to remove all the keyframes from the root bone in blender, and then rotate is same bone.

Unfortunetely, I'm late with some work to deliver, so i cant record a video for you, but I show this in some tutorials I've done in the past.

Found it. in this moment I explain visually what I mean bu deleting everything from the root bone and rotate it. https://youtu.be/S8nSDZjcaSA?t=3180

Hope it helps

[0Xiao0]

@carlosedubarreto thanks a lot , I will try it

[lvZic]

Hello @0Xiao0 , I mean in blender.

If you use the script to import the easymocap data in blender, by default, the root bone (the parent of all bones) will be keyframed.

The easiet way that I found to explain to rotate the character is to remove all the keyframes from the root bone in blender, and then rotate is same bone.

Unfortunetely, I'm late with some work to deliver, so i cant record a video for you, but I show this in some tutorials I've done in the past.

Found it. in this moment I explain visually what I mean bu deleting everything from the root bone and rotate it. https://youtu.be/S8nSDZjcaSA?t=3180

Hope it helps

hi, i wonder if there is anyway to cvt the data in code? I met this problem because there is difference betweem SMPL and Mixamo model. I donot want to modify Mixamo model. So the SMPL's axis-angles should be changed.

[0Xiao0]

Hello @0Xiao0 , I mean in blender. If you use the script to import the easymocap data in blender, by default, the root bone (the parent of all bones) will be keyframed. The easiet way that I found to explain to rotate the character is to remove all the keyframes from the root bone in blender, and then rotate is same bone. Unfortunetely, I'm late with some work to deliver, so i cant record a video for you, but I show this in some tutorials I've done in the past. Found it. in this moment I explain visually what I mean bu deleting everything from the root bone and rotate it. https://youtu.be/S8nSDZjcaSA?t=3180 Hope it helps

hi, i wonder if there is anyway to cvt the data in code? I met this problem because there is difference betweem SMPL and Mixamo model. I donot want to modify Mixamo model. So the SMPL's axis-angles should be changed.

sorry,i have no idea, too

[thefudgeishot]

If I remember correctly both method goes through blender, the difference would be the process done inside of blender and not the output itself. I probably referred as BVH and FBX to tell which method I used.

@carlosedubarreto another problem. When I imported the fbx file in Blender, strange results appear like image below however, the demo result seems normal:

Got the same issue, however @carlosedubarreto 's suggested fix doesn't seem to be working for me, no amount of rotating the rig has seemed to work.

Could there potentially be something wrong with the script?

[carlosedubarreto]

@thefudgeishot , can you do a quick record of the process you are doing after running the import in blender, so I can see how you are doing things?

[carlosedubarreto]

Hello @0Xiao0 , I mean in blender. If you use the script to import the easymocap data in blender, by default, the root bone (the parent of all bones) will be keyframed. The easiet way that I found to explain to rotate the character is to remove all the keyframes from the root bone in blender, and then rotate is same bone. Unfortunetely, I'm late with some work to deliver, so i cant record a video for you, but I show this in some tutorials I've done in the past. Found it. in this moment I explain visually what I mean bu deleting everything from the root bone and rotate it. https://youtu.be/S8nSDZjcaSA?t=3180 Hope it helps

hi, i wonder if there is anyway to cvt the data in code? I met this problem because there is difference betweem SMPL and Mixamo model. I donot want to modify Mixamo model. So the SMPL's axis-angles should be changed.

@lvZic , I think that converting in code is waycomplex than the way I suggested. On recent experiments, after a lot of problems, I was able to rotate it with success on my newwest addon, but to do that I had to use a lot of tricks using constraints and baking them (if I remember correctly)

[lvZic]

Hello @0Xiao0 , I mean in blender. If you use the script to import the easymocap data in blender, by default, the root bone (the parent of all bones) will be keyframed. The easiet way that I found to explain to rotate the character is to remove all the keyframes from the root bone in blender, and then rotate is same bone. Unfortunetely, I'm late with some work to deliver, so i cant record a video for you, but I show this in some tutorials I've done in the past. Found it. in this moment I explain visually what I mean bu deleting everything from the root bone and rotate it. https://youtu.be/S8nSDZjcaSA?t=3180 Hope it helps

hi, i wonder if there is anyway to cvt the data in code? I met this problem because there is difference betweem SMPL and Mixamo model. I donot want to modify Mixamo model. So the SMPL's axis-angles should be changed.

@lvZic , I think that converting in code is waycomplex than the way I suggested. On recent experiments, after a lot of problems, I was able to rotate it with success on my newwest addon, but to do that I had to use a lot of tricks using constraints and baking them (if I remember correctly)

Thanks for your reply. I have used retargeting to do it.

[thefudgeishot]

@thefudgeishot , can you do a quick record of the process you are doing after running the import in blender, so I can see how you are doing things?

Sure I followed the insturctions you had provided previously

To solve this what I do is get the root bone, remove all the keyframes, and rotate it 90 degrees on the y axis. remove all the frames only from the root bone.

Also rotated it on the x axis. In this case I also rotated it by -90 degrees on the y axis.

In case its relevant I imported the model with File>Import>FBX

Video: https://youtu.be/yxLy4Gs05G8

Thanks, for looking into this

[carlosedubarreto]

@thefudgeishot , can you do a quick record of the process you are doing after running the import in blender, so I can see how you are doing things?

Sure I followed the insturctions you had provided previously

To solve this what I do is get the root bone, remove all the keyframes, and rotate it 90 degrees on the y axis. remove all the frames only from the root bone.

Also rotated it on the x axis. In this case I also rotated it by -90 degrees on the y axis.

In case its relevant I imported the model with File>Import>FBX

Video: https://youtu.be/yxLy4Gs05G8

Thanks, for looking into this

I feel that you are doing almost everything right. I have a feeling that the problem is the fbx used for the SMPL.

And I dont remember if the correct on is the SMPL or SMPL-X. The order of the bones between each other is a little bit different, if I'm remembering it all correctly;

[thefudgeishot]

I feel that you are doing almost everything right. I have a feeling that the problem is the fbx used for the SMPL.

And I dont remember if the correct on is the SMPL or SMPL-X. The order of the bones between each other is a little bit different, if I'm remembering it all correctly;

Right okay, I tried to use the SMPL-X unity package as well as the SMPL unity package and neither seemed to do much to rectify the issue. Could I potentially be using the wrong package entirely?

[carlosedubarreto]

@thefudgeishot , I think there is a possiblity that you are using the wrong package.

Its a while since I use this solution.

I would suggest you to take a look on what I suggested last year in this same thread.

Here is the link to the message https://github.com/zju3dv/EasyMocap/issues/10#issuecomment-1139052314

This link shows 2 solutions I made to make it easier for people to use easymocap. since you already have your result, you could try mocap import

[thefudgeishot]

@thefudgeishot , I think there is a possiblity that you are using the wrong package.

Its a while since I use this solution.

I would suggest you to take a look on what I suggested last year in this same thread.

Here is the link to the message

https://github.com/zju3dv/EasyMocap/issues/10#issuecomment-1139052314

This link shows 2 solutions I made to make it easier for people to use easymocap.

since you already have your result, you could try mocap import

Thanks for the suggestions, would it be possible to provide the link to correct package? I'm trying to use easymocap in a project that requires cli, and not a graphical interface like you've suggested. Sorry if this causes too much of a inconvenience.

[carlosedubarreto]

I was suggesting the addon to see what is used over there. because the solution used on the script shared in this issue, should be the same used on the addon.

The problem is that I'm so busy with work, that I cannot stop to look there to tell you. I could give the wrong indication.

[qiffang]

@carlosedubarreto @chingswy The length of poses in output of smpl is 69, not 72. Can you tell me why, please

[
    {
        "id": 0,
        "Rh": [
          [-3.033, -0.206, 0.180]
        ],
        "Th": [
          [-0.484, -0.096, 3.765]
        ],
        "poses": [
          [-0.926, 0.231, 0.141, 0.397, -0.398, -0.122, 0.228, 0.180, 0.008, 1.445, 0.900, -0.454, -0.684, 0.084, -0.206, -0.087, -0.175, -0.019, 0.002, -0.049, -0.102, 0.360, -0.332, 0.589, 0.065, -0.035, 0.047, -0.377, 0.267, 0.003, -0.280, 0.569, -0.904, -0.236, 0.325, -0.170, -0.150, -0.477, -0.170, 0.008, 0.128, 0.260, 0.198, 0.463, 0.133, -0.298, -0.855, -0.579, 0.127, -0.029, 0.645, -0.003, -0.847, 0.046, -0.162, 0.351, 0.139, -0.082, -0.167, 0.375, 0.210, 0.050, -0.435, 0.214, -0.194, -0.148, -0.154, 0.223, 0.347]
        ],
        "shapes": [
          [0.493, 0.502, -0.499, 0.506, -0.353, 0.001, -0.003, 0.403, 0.000, 0.250]
        ]
    }
]

[carlosedubarreto]

@qiffang if i remember correctly, you have to apoend the global position, or something like that.

Its been a while that i dint look at this code

[qiffang]

@carlosedubarreto Thanks for your response. Can you tell me how to append the global position? is it use_root_rot flag? Thanks very much

[qiffang]

@carlosedubarreto another question, does easymocap support to remove original images in render.mp4?

[carlosedubarreto]

@qiffang sorry its been a while , so i really dont remember. But I looked on the code and I think this is the part that I was mentioning.

What I can suggest you, is to look on some solution that already works.

I have at least 3 addons for blender that works with easymocap code.

https://carlosedubarreto.gumroad.com/l/ceb_easymocap_multicam?layout=profile https://carlosedubarreto.gumroad.com/l/ceb_em_monocular?layout=profile https://carlosedubarreto.gumroad.com/l/mocap_import?layout=profile

you can get them all for free (if there is some that has a price, you will find a code on the page to get it for free.)