Need some help on getting Frankmocap Data onto Auto-Rig Pro default biped rig

[FrankSpalteholz]

Dear Carlos,

first of all i'd like to thank you for implementing a pipe between Blender and Frankmocap! But because i'm way more familiar with Maya and a fresh noob to Blender i'd like to ask for some guidance on how to get this to work. So here is my progress so far:

1. Running Frankmocap on Linux and Windows -> exporting anim as .mkl sequence (Done)
2. Using Blender 2.90.1 (installed Auto-Rig Pro) and also your Add-on (Done)
3. Creating a default human biped-rig with ARP (Done)
4. Trying to import mkl-sequence via Mocap-UI (your add-on) Fails

When i browse to my folder with the mkl files and select the first one it got this error:

Traceback (most recent call last): File "C:\Users\frank\AppData\Roaming\Blender Foundation\Blender\2.90\scripts\addons\b3d_mocap_import-main\load_mocap.py", line 729, in execute import joblib ModuleNotFoundError: No module named 'joblib'

location: :-1

It's obvisously due to a missing "joblib"-package and i've already tried the button "Install Joblib" in the UI but no luck. So again ... if you could give me small notes on how to get the data loaded to the rig i'd be super thankful. Also maybe i was missing some steps inbetween.

Thank you very much!

[carlosedubarreto]

That´s odd. Frannkmocap needs PKL files, not MKL. and joblib is needed for Vibe, not for frankmocap.

To install joblib you can use this script (replacing the folder with you blender installation directory

D:\Blender\blender-2.92.0-windows64\2.92\python\bin\python.exe D:\Blender\blender-2.92.0-windows64\2.92\python\lib\site-packages\pip install joblib'

The "D:\Blender\blender-2.92.0-windows64\2.92" is where it is in my machine, you replace this with the folder where your blender is

[FrankSpalteholz]

Hi Carlos, my bad ... the sequence is ofcourse a .pkl sequence not mkl. Sorry for that But overall my steps are correct? Do i have to select anything on the ARP-rig before applying the data like the whole joint-structure or the root? Thanks again!

[carlosedubarreto]

Something is odd in your process, but let's go in parts.

Did you solve the error?

When you run the import of frankmocap data it should show a new armature.

Does It show up?

[FrankSpalteholz]

Installed Joblib within Blender-sitepackages ... created a clean scene (nothing in there) and hit import. Now i got this error: Traceback (most recent call last): File "C:\Users\frank\AppData\Roaming\Blender Foundation\Blender\2.90\scripts\addons\b3d_mocap_import-main\load_mocap.py", line 1070, in execute for limb in range(len(nppic['pred_output_list'][0]['pred_body_joints_img'])): KeyError: 'pred_body_joints_img'

location: :-1

Edit: i've used this snippet for exporting (body-mocap-only) from frankmocap: python -m demo.demo_bodymocap --input_path ./sample_data/han_short2.mp4 --out_dir ./mocap_output --save_pred_pkl

[FrankSpalteholz]

Ok i made some promissing progress. The error i've made was, to use the body_mocap demo instead fo the full frank_mocap one. Now its importing the armature in Blender which is great. There is also animation applied to. 2 things i'm still wondering of. 1. The rig is placed somewhere really low and moving up duiing animation as if it's been pulled by a string. is this a common behaviour?

2. Also there are no fingers added Thank again!

[carlosedubarreto]

Yes it is normal to the animation be in the wrong place. But it is simple to solve. Grab the root bone, remove its keyframes and place it where you want.

About the fingers. I didnt add it, because I was not going to work well, So I didnt go forward with that. The only solution I've made, in blender to import finger animation is using easymocap.

[FrankSpalteholz]

Thank you for your fast support!

[carlosedubarreto]

Lol, and I thought I took too long to answer.

BTW, I want to do some more work on frankmocap in the future, but aftaer i finish some major things I'm working on, like the facial mocap and a workfloe to integrate the full body animation with easymocap and the facial animation.

[Faizan-Saifullah]

Dear Carlos,

I am importing frankmocap using your addon for blender. I have done it but I have two issues:

1. How to add bones for hands.
2. My rig keeps on rotating and performing different because of that than the animation generated by frankmocap. It appears as if the rig is locked at some point (probably hip).

Can you guide?

Regards

[carlosedubarreto]

Hello @Faizan-Saifullah ,

It's been a long time since I dont try frankmocap, I think when I used it, there was only the mocap data for the body. To add hands I would suggest you to try easymocap, they have hands control too.

About the hips movement, I know there are some scripts that do not create this mevement, I'm sorry again, but it has at least 2 yeasrs that I dont use frankmocap.

In any case, have you tried easymocap, it has the multicamera and the monocular solution. I found them much better than most solutions, that why I didnt use frankmocap or vibe anymore.

[Faizan-Saifullah]

Actually, we are using frankmocap because the results on monocular camera are really good minus some jittering and locking issues and even better than DeepMotion's Animate3D. I will try EasyMocap also. Thanks.

[carlosedubarreto]

if you allow me to suggest, there is this addon I've made, using easymocap monocular, that I think will be more hepfull to you. https://carlosedubarreto.gumroad.com/l/ceb_em_monocular

you can get it for free, just insert the coupon on the description

[Faizan-Saifullah]

@carlosedubarreto Thank you. Yes, I have checked it but it doesn't have hands or does it?

[Faizan-Saifullah]

Here, the hand was folded in original video. Is there any way to add hands or is there something that I missing to add hands?

[carlosedubarreto]

The only free solution that works very well that I know is the easymocap multicam, you can run it and have the hands working too.

you can see a simple test (not the best one) in this link https://www.youtube.com/watch?v=_ELM-VQMIGo