Export Animated bvh or FBX?

[huh8686]

First, thanks for your great work.

Just wondering if exporting animated format(such as bvh or fbx) is supported ? Looks like demo.py only exports T-pose skeleton . I've tried to combine all outputs to fbx format but had trouble with animating character with a given pose. (In blender) I used pose (FrameN, Joint_Num,3) data as an input of euler value but it doesn't work.

Any Suggestions??

[yangtao19920109]

your pose data must format as SMPL data,and you can edit the write bvh code to save bvh file

[PeizhuoLi]

Hi, the pose file is not of euler angle but of angle-axis representation. You may need to convert it to euler angle before putting it into Blender. We'll consider your suggestion and output the animated bvh as well in our next update (estimated in mid June).

[huh8686]

I've implemented a script to combine all output(e.g. T-pose.obj, weight.npy, skeleton.bvh) to "FBX /gltf format or animated bvh"

In the script , 1) pose data are converted to Quaternion data . 2) skeleton.bvh -> smpl rigging model (This is because applying animation to skeleton.bvh directly cause an error whereas smpl rigging model works well - Guessing this is because of blender rigging system.)

-skeleton.bvh- -smpl rigging model-

Still there is an issue that obj sequence and fbx are not perfectly matched

I couldn't figure out why there is a difference between two model.

Is there anything I've missed??

e.g.)

Attached my script here

#USAGE : blender -b -P nbs_fbx_output.py -- --input ../demo --output ../demo/output.fbx --pose ../eval_constant/sequences/greeting.npy

import bpy

import numpy as np
from mathutils import Matrix,Vector,Quaternion

import os
import sys
import argparse

class ArgumentParserForBlender(argparse.ArgumentParser):
    """
    This class is identical to its superclass, except for the parse_args
    method (see docstring). It resolves the ambiguity generated when calling
    Blender from the CLI with a python script, and both Blender and the script
    have arguments. E.g., the following call will make Blender crash because
    it will try to process the script's -a and -b flags:
    >>> blender --python my_script.py -a 1 -b 2

    To bypass this issue this class uses the fact that Blender will ignore all
    arguments given after a double-dash ('--'). The approach is that all
    arguments before '--' go to Blender, arguments after go to the script.
    The following calls work fine:
    >>> blender --python my_script.py -- -a 1 -b 2
    >>> blender --python my_script.py --
    """

    def _get_argv_after_doubledash(self):
        """
        Given the sys.argv as a list of strings, this method returns the
        sublist right after the '--' element (if present, otherwise returns
        an empty list).
        """
        try:
            idx = sys.argv.index("--")
            return sys.argv[idx+1:] # the list after '--'
        except ValueError as e: # '--' not in the list:
            return []

    # overrides superclass
    def parse_args(self):
        """
        This method is expected to behave identically as in the superclass,
        except that the sys.argv list will be pre-processed using
        _get_argv_after_doubledash before. See the docstring of the class for
        usage examples and details.
        """
        return super().parse_args(args=self._get_argv_after_doubledash())
def init_scene():
    bpy.ops.object.select_all(action='SELECT')
    bpy.ops.object.delete(use_global=False)

def import_obj(filepath):
    bpy.ops.import_scene.obj(filepath=filepath,split_mode="OFF")

def import_skeleton(filepath):
    bpy.ops.import_anim.bvh(filepath=filepath, filter_glob="*.bvh", target='ARMATURE', global_scale=1, frame_start=1, use_fps_scale=False, use_cyclic=False, rotate_mode='NATIVE', axis_forward='-Z', axis_up='Y')

def set_pose_from_rodrigues(armature, bone_name, rodrigues):
    rod = Vector((rodrigues[0], rodrigues[1], rodrigues[2]))
    angle_rad = rod.length
    axis = rod.normalized()

    if armature.pose.bones[bone_name].rotation_mode != 'QUATERNION':
        armature.pose.bones[bone_name].rotation_mode = 'QUATERNION'

    quat = Quaternion(axis, angle_rad)

    armature.pose.bones[bone_name].rotation_quaternion = quat

    return

def export_animated_mesh(output_path,armature,mesh):
    # Create output directory if needed
    output_dir = os.path.dirname(output_path)
    if not os.path.isdir(output_dir):
        os.makedirs(output_dir, exist_ok=True)

    # Select only skinned mesh and rig
    bpy.ops.object.select_all(action='DESELECT')
    armature.select_set(True)
    mesh.select_set(True)

    if output_path.endswith('.glb'):
        print('Exporting to glTF binary (.glb)')
        # Currently exporting without shape/pose shapes for smaller file sizes
        bpy.ops.export_scene.gltf(filepath=output_path, export_format='GLB', export_selected=True, export_morph=False)
    elif output_path.endswith('.fbx'):
        print('Exporting to FBX binary (.fbx)')
        bpy.ops.export_scene.fbx(filepath=output_path, use_selection=True, add_leaf_bones=False)

    elif output_path.endswith('.bvh'):
        print('Exporting to BVH (.bvh)')
        bpy.ops.export_anim.bvh(filepath=output_path, check_existing=True, filter_glob='*.bvh', global_scale=1.0, rotate_mode='NATIVE', root_transform_only=False)
    else:
        print('ERROR: Unsupported export format: ' + output_path)
        sys.exit(1)

    return

if __name__ == '__main__':
    try:
        if bpy.app.background:

            parser = ArgumentParserForBlender()

            parser.add_argument('--input', dest='input_dir', type=str, required=True,
                                help='Input directory')
            parser.add_argument('--output', dest='output_path', type=str, required=True,
                                help='Output file or directory')

            parser.add_argument('--pose', type=str, default=None,
                                help='Always use specified gender')

            args = parser.parse_args()  

            input_dir = args.input_dir
            output_path = args.output_path
            pose_path = args.pose

            if pose_path is None:
                exportAnimation  = False
            else :
                exportAnimation = True

            obj_path = os.path.join(input_dir,'T-pose.obj')
            skeleton_path = os.path.join(input_dir,'skeleton.bvh')
            weight_path = os.path.join(input_dir,'weight.npy')

            init_scene()

            import_obj(obj_path)
            mesh = bpy.context.selected_objects[0]

            import_skeleton(skeleton_path)
            skeleton = bpy.context.selected_objects[0]

            mesh.select_set(False)
            skeleton.select_set(True)
            bpy.context.view_layer.objects.active = skeleton

            bpy.ops.object.mode_set(mode='EDIT')

            for bone in skeleton.data.edit_bones:

                bone.use_connect = False

                target_vec = Vector((0,0,-1))

                vec = Vector(bone.head - bone.tail)

                rot = vec.rotation_difference(target_vec).to_matrix().to_4x4()

                R = (Matrix.Translation(bone.head) @
                     rot @
                     Matrix.Translation(-bone.head)
                    )
                bone.transform(R) 

            bpy.ops.object.mode_set(mode='OBJECT')

            mesh.select_set(True)
            skeleton.select_set(True)
            bpy.context.view_layer.objects.active = skeleton

            bpy.ops.object.parent_set(type='ARMATURE_AUTO')

            weight = np.load(weight_path)

            for vg in range(weight.shape[1]):
                vg_index = str(vg).zfill(2)
                for i in range(weight[:,vg].shape[0]):
                    if weight[:,vg][i]> 0:
                        mesh.vertex_groups[vg_index].add([i], weight[:,vg][i], 'REPLACE')

            #Set Animation 
            if exportAnimation:
                anim = np.load(pose_path)
                poses = anim[:, :-3]
                locs = anim[:,-3:]

                poses = poses.reshape(poses.shape[0], -1, 3)

                for frame in range(poses.shape[0]):

                    for j in range(poses.shape[1]):
                        if j==0:
                            continue
                            #NOT SURE ABOUT LOCATION
                            #skeleton.pose.bones['00'].location = locs[frame]
                            #skeleton.pose.bones['00'].keyframe_insert('location', frame=frame)

                        bone_name = str(j).zfill(2)

                        set_pose_from_rodrigues(skeleton, bone_name, poses[frame][j])  

                        skeleton.pose.bones[bone_name].keyframe_insert('rotation_quaternion', frame=frame)

            export_animated_mesh(output_path,skeleton,mesh) 
    except SystemExit as ex:
        if ex.code is None:
            exit_status = 0
        else:
            exit_status = ex.code

        print('Exiting. Exit status: ' + str(exit_status))

        # Only exit to OS when we are not running in Blender GUI
        if bpy.app.background:
            sys.exit(exit_status)

[huh8686]

Still have some issues...

Could you share some scripts that I can generate pose.npy (e.g. ./eval_constant/sequence/dance.npy) It would be helpful to fix some errors ..

[PeizhuoLi]

Hi, really sorry for my late reply. I was too occupied in the past weeks : (

I updated the code to support output the animated bvh file with the parameter --animated_bvh=1. Now I think it should work with your code (without exportAnimation section).

And thank you for sharing your amazing code with everyone! You are welcome to start a pull request to add your code as part of the repo : )

Let me know if there is any other issue.

[huh8686]

Thanks to animated bvh , I could solve the issues easily. Simply I've just used animated bvh to rig , instead of calculating rotation of joints which cause many issues due to the rigging system in blender.

Furthermore , I've add some functions to generate neural-blendshapes model. (Only envelope-only model was generated from my previous script)

To do so, "coff.npy" and "basis.npy" should be exported first. I add the code below in the forward function of ./architecture/blend_shapes.py

np_basis = basis_full.detach().cpu().numpy()
np_coff = coff.detach().cpu().numpy()
np.save('./demo/basis.npy',np_basis)
np.save('./demo/coff.npy',np_coff)

Result (Comparison between obj seq and fbx)

I've requested a pull as well. (nbs_fbx_output.py)

Closing this issue as it's solved already

[PeizhuoLi]

Thanks again for your fantastic code! I've merged your pull request and updated the code for saving blend shapes stuff and README.md.