Support `--save-pk` in demo_video_x.py

[orenwang]

Currently the implementation of --save-pk is missing (and incorrectly named as "--save-pt") in demo_video_x.py.

This PR adds this implementation and outputs a res.pk file similar to demo_video.py.

However, if anyone wants to import the pickle file into Blender, you will have to replace files in the original add-on with these hybrik_blender_addon_modified_files that I modified for SMPLX models (make sure to choose "Smplx Neutral" as your model before importing the pickle file).

[alevonian]

Thank you for this additional work, especially since the existing commented code in demo_video_x.py was left commented out and buggy. While your changes indeed generate the pickle file, using your changes to the blender add on only has a single frame from the generated dance video when choosing male or female, and when choosing "Simplx Neutral", it imports nothing.

Perhaps I am missing something? Thanks again!

[orenwang]

Did you download the Smplx Neutral then select 'Smplx Neutral' before importing the pickle? @alevonian You will have to put the SMPLX model at the same location as the SMPL male and female models inside the Add-on script. Also, you will need to reload the Add-on but i don't how familiar you are with Blender Add-ons. If you don't know how to reload, just zip a new file and re-install it.

[alevonian]

Hi Orenwang, Thanks for the quick reply. I added the SMPLX_NEUTRAL.npz and SMPLX_NEUTRAL.pkl inside the hybrik_blender_addon.zip, inside the same directory (hybrik_blender_addon) where you had your mods (init.py and convert2bvh.py), and readded the plugin, but I get the same result, no geometry when choosing simplx neutral and importing the pickle file. I also added the male and female there, with the same result. Inside the simplx directory there is also the "smplx_npz.zip" file, so perhaps that needs to be added?

Nope, that didn't work either.

[orenwang]

Does this make it clearer?

hybrik_blender_addon
│   __init__.py
│   convert2bvh.py
│   README.md
│
└───data
│   │   basicModel_f_lbs_10_207_0_v1.0.2.fbxt
│   │   basicModel_m_lbs_10_207_0_v1.0.2.fbx
│   │   smplx-neutral.fbx

You need to use the fbx format model, same as the female and male models used by the original add-on provided in this repo.

[alevonian]

Yes, that is what I figured, but I have to find smplx-neutral.fbx somewhere. :) I searched everywhere on the internet, no luck.

[orenwang]

Just download it in the SMPLX page. Due to licensing, no one can distribute it directly.

[alevonian]

I think I found it in the Unity zip.

[alevonian]

Hah, ok, it is working now. It took like 3 minutes for Blender to finish processing the pickle file, but it worked. Thank you so much!

[alevonian]

The bones are incorrectly oriented and the model jumps all over the place, so there may be additional settings to correct for that, but at least it animates. "Reset Locations" helps a little.

[stevechaw]

Just download it in the SMPLX page. Due to licensing, no one can distribute it directly.

Hi, there~ Nice work! I use your code and run well. But there is a problem about fingers. The last joint of every finger will up after extract bvh in blender. Can you check out this issue? Thanks!

[BobHop]

Currently the implementation of --save-pk is missing (and incorrectly named as "--save-pt") in demo_video_x.py.

This PR adds this implementation and outputs a res.pk file similar to demo_video.py.

However, if anyone wants to import the pickle file into Blender, you will have to replace files in the original add-on with these hybrik_blender_addon_modified_files that I modified for SMPLX models (make sure to choose "Smplx Neutral" as your model before importing the pickle file).

Thanks for your work on implementing / putting back .pkl export for HybrIK-X! I've finally been able to get something useful out of it, it's amazing. :) I'm grateful for the updated Blender addon files as well. Now, when comparing HybrIK-X's raw output and the imported .pkl in Blender, the hands seem to suffer quite a bit... Do you have any idea why? The fingers are hooked and distorded, which is a pity because you can still discern the captured hand poses underneath.

So yeah, any help or clue regarding a way to fix this would be awesome.

[orenwang]

@BobHop it's hard to tell what goes wrong without more info. Maybe some screenshots, or reproducible files?

[BobHop]

Indeed. Sorry for the lack of info. Besides, I found a way to correct it! Here are four screenshots to better describe the problem:

1. Original picture
2. Same with raw HybrIK-X overlay
3. Imported .pk animation in Blender, default smplx-neutral.fbx (taken from SMPL-X Unity package)
4. Same imported .pk file, updated smplx-neutral.fbx (exported from SMPL-X Blender Add-on)

Having read the previous comments in this discussion, initially I took the smpl-neutral.fbx file from SMPL-X's Unity package; the result is what you see in picture number 3. Later on, I had a look at SMPL-X's Blender Add-on because it features FBX export. So I tried... In an empty scene in Blender, using SMPL-X's addon (the '300 shape components' version) I added the SMPLX_NEUTRAL model and exported it as an .fbx, overwriting the one in the data folder of HybrIK-X's addon. I then tried another import of the same .pk animation and got what you seen in picture number 4. Fingers fixed!!

The hand poses aren't perfect, of course – but they're identical to the HybrIK-X raw output, so I guess that's the best fidelity we can get using this project. To me, it's still one of the best methods because HybrIK-X requires only one camera, uses less VRAM yet is rather fast, can work with cropped bodies to some extent... You don't get as much accuracy as other methods, especially for the hands, but it's really ok for simple animation needs.

[BobHop]

PS : Oddly, the "Export FBX" button in SMPL-X's Blender add-on produces an .fbx file that doesn't fix the hands (i.e. by clicking the "Update Joint Locations", which is how I obtained result number 4). But using Blender's export to FBX, does produce a "better" .fbx for that matter. The SMPL-X add-on offers some options I didn't try though: I exported "for Unity" but there's also a "for Unreal" choice; maybe this would give different results.