Ability to import MakeHuman BVH-style poses

[Vidyut]

I have downloaded some poses and the plugin has a load pose section, but I can't figure out what exactly I have to do.

[joepal1976]

The short answer is that MPFB does not support MH poses yet. There is a separate pose system for MPFB, but the online repository is for MH-style poses.

The longer technical answer is that MH use poses defined as BVH, which is a very old and rather complicated format. While it would be theoretically possible to parse a BHV style pose file and apply it to a skeleton in Blender, I've so far failed to get this to work in any acceptable way.

You can open the poses in blender (file -> import bvh, use "Y forward, Z up") :

I guess it'd be possible to do something akin to the mixamo snapping functionality to use a MH style pose this way.

[joepal1976]

Actually, I think I'll take a swipe at going down that router. It'd be an interesting experiment.

[Vidyut]

This sounds great, but I didn't mean BVH - there are community assets with "mhpose" extension and also expressions. Then there's a "load pose" panel in the addon - so I thought ... it does something.

Anyway, what you are planning to do sounds amazing. I just love making suggestions here. You manage to make sense of gibberish and come up with useful upgrades. Very superb.

[joepal1976]

What is called "poses" in the online repos are BVH files, see for example http://www.makehumancommunity.org/content/grabbed_by_the_neck.html, whereas what is called "expressions" are mhpose files, for example http://www.makehumancommunity.org/content/smug_grin.html

Expressions are defined in yet another pose system. Its root are "pose units" defined in https://github.com/makehumancommunity/makehuman/blob/master/makehuman/data/poseunits/face-poseunits.json, which in turn refers to a BVH file. In the BVH file, each frame defines what one might call microexpressions or atomic face movements, such as lifting an eyebrow. A mhpose expression then lists how much of each such microexpression should be used, as in for example https://github.com/makehumancommunity/makehuman-assets/blob/master/base/expressions/anger01.mhpose

Porting the expression system is a lot further away than being able to import poses. Since the big file with micro expressions would have to be ported first somehow.

[Vidyut]

Oh. Oooookay.

I guess I could try a BVH plugin and see what happens, but it may end up being simpler to pose the characters directly.

I googled and found BVH to fbx "converters" - that could help in a more Mixamo approach.

[joepal1976]

Well, initial experiments shows that it is not as easy as simply copying the rotation of the bones in an imported BVH file.

Left is imported BVH, right is MPFB default rig where each bone has copied the rotation of the bone with the same name in the BVH:

My guess is that there is something related to rotation mode that causes this, but I have no clue what as of yet.

[Vidyut]

I found this. https://github.com/mcsantiago/bvh2fbx Perhaps it might work or could be modified or give you ideas for how it handles rotation or etc

[joepal1976]

It is possible that I'm "almost" there. I've identified the problem, but don't know how to solve it.

It all boils down to the BVH and MPFB armatures having different bone rolls. The BVH bone roll is consistently 0.0, while the MPFB bone roll is set for bones to rotate in a predictable manner. For example the bone "shoulder01.L" has a bone roll of 117.

This is the code I have so far:

    @staticmethod
    def import_bvh_file_as_pose(dest_rig, bvh_file_path):
        """Import a bvh file as a pose for the given armature."""

        if not os.path.exists(bvh_file_path):
            _LOG.error("bvh_file_path does not exist", bvh_file_path)
            raise IOError("BVH file does not exist " + bvh_file_path)

        ObjectService.activate_blender_object(dest_rig, deselect_all=True)
        bpy.ops.object.mode_set(mode='EDIT', toggle=False)

        dest_rolls = dict()
        source_rolls = dict()

        for dest_edit_bone in dest_rig.data.edit_bones:
            dest_rolls[dest_edit_bone.name] = dest_edit_bone.roll

        bpy.ops.object.mode_set(mode='POSE', toggle=False)

        bpy.ops.import_anim.bvh(filepath=bvh_file_path, axis_forward='Y', axis_up='Z', rotate_mode='XYZ')
        source_rig = bpy.context.object
        _LOG.debug("source_rig", source_rig)

        ObjectService.activate_blender_object(source_rig, deselect_all=True)
        bpy.ops.object.mode_set(mode='EDIT', toggle=False)

        for source_edit_bone in source_rig.data.edit_bones:
            source_rolls[source_edit_bone.name] = source_edit_bone.roll

        bpy.ops.object.mode_set(mode='POSE', toggle=False)

        for source_pose_bone in source_rig.pose.bones:
            dest_pose_bone = RigService.find_pose_bone_by_name(source_pose_bone.name, dest_rig)

            # _LOG.debug("Pose bones (bvh, armature)", (source_pose_bone, dest_pose_bone))

            if dest_pose_bone:
                source_roll = source_rolls[source_pose_bone.name]
                dest_roll = dest_rolls[dest_pose_bone.name]

                roll_difference = dest_roll - source_roll

                _LOG.debug("Rolls (name, bvh, armature, difference)", (dest_pose_bone.name, source_roll, dest_roll, math.degrees(roll_difference)))

                # -- NOW WHAT? --

With the difference in bone roll, copying the rotation from one bone to the other ends up wrong. Sadly, my math is too sub par to figure out how to compensate for the roll difference. It'd probably involve something related to a matrix, which in my world is mostly guys in long black robes fighting computers while a lot of green letters are flowing down the screen.

[joepal1976]

Anyway, I'll give up for now. I'll need some help with this.

I've pushed what I have, and the code is in the top of animationservice.py.

[ksami]

Could this article help? https://diffeomorphic.blogspot.com/p/bvh-how-it-works.html Don't fully understand it but it looks like a similar problem.

Also interestingly, I found that from #33 when I was messing around with BVH animations previously.

[joepal1976]

Thanks for the suggestions. I have been messing around with different strategies for compensating for roll, but so far failed to find something that actually works.

I did, however, manage to conceive a strategy which actually manages to load a MH-style BVH pose on a default rig:

This works in the sense that you get a character put in the desired pose. However, all the bone roll values have then been overwritten, making further posing cumbersome. But if you're happy with the static pose, then this is a functional solution.

I'll consider this good enough for now.

[joepal1976]

This is in github master now and will appear in tomorrow's nightly build.

[Vidyut]

It may not be ideal, but it will probably solve two use cases for me - posing for stills and for very short clips without much movement.

Also I really must say this again. One of the biggest reasons I love MPFB is how talented and quick you are with making our work easier. Thanks a lot. Astonishing in the best way.