Closed huh8686 closed 3 years ago
your pose data must format as SMPL data,and you can edit the write bvh code to save bvh file
Hi, the pose file is not of euler angle but of angle-axis representation. You may need to convert it to euler angle before putting it into Blender. We'll consider your suggestion and output the animated bvh as well in our next update (estimated in mid June).
I've implemented a script to combine all output(e.g. T-pose.obj, weight.npy, skeleton.bvh) to "FBX /gltf format or animated bvh"
In the script , 1) pose data are converted to Quaternion data . 2) skeleton.bvh -> smpl rigging model (This is because applying animation to skeleton.bvh directly cause an error whereas smpl rigging model works well - Guessing this is because of blender rigging system.)
-skeleton.bvh-
-smpl rigging model-
Still there is an issue that obj sequence and fbx are not perfectly matched
I couldn't figure out why there is a difference between two model.
Is there anything I've missed??
e.g.)
Attached my script here
#USAGE : blender -b -P nbs_fbx_output.py -- --input ../demo --output ../demo/output.fbx --pose ../eval_constant/sequences/greeting.npy
import bpy
import numpy as np
from mathutils import Matrix,Vector,Quaternion
import os
import sys
import argparse
class ArgumentParserForBlender(argparse.ArgumentParser):
"""
This class is identical to its superclass, except for the parse_args
method (see docstring). It resolves the ambiguity generated when calling
Blender from the CLI with a python script, and both Blender and the script
have arguments. E.g., the following call will make Blender crash because
it will try to process the script's -a and -b flags:
>>> blender --python my_script.py -a 1 -b 2
To bypass this issue this class uses the fact that Blender will ignore all
arguments given after a double-dash ('--'). The approach is that all
arguments before '--' go to Blender, arguments after go to the script.
The following calls work fine:
>>> blender --python my_script.py -- -a 1 -b 2
>>> blender --python my_script.py --
"""
def _get_argv_after_doubledash(self):
"""
Given the sys.argv as a list of strings, this method returns the
sublist right after the '--' element (if present, otherwise returns
an empty list).
"""
try:
idx = sys.argv.index("--")
return sys.argv[idx+1:] # the list after '--'
except ValueError as e: # '--' not in the list:
return []
# overrides superclass
def parse_args(self):
"""
This method is expected to behave identically as in the superclass,
except that the sys.argv list will be pre-processed using
_get_argv_after_doubledash before. See the docstring of the class for
usage examples and details.
"""
return super().parse_args(args=self._get_argv_after_doubledash())
def init_scene():
bpy.ops.object.select_all(action='SELECT')
bpy.ops.object.delete(use_global=False)
def import_obj(filepath):
bpy.ops.import_scene.obj(filepath=filepath,split_mode="OFF")
def import_skeleton(filepath):
bpy.ops.import_anim.bvh(filepath=filepath, filter_glob="*.bvh", target='ARMATURE', global_scale=1, frame_start=1, use_fps_scale=False, use_cyclic=False, rotate_mode='NATIVE', axis_forward='-Z', axis_up='Y')
def set_pose_from_rodrigues(armature, bone_name, rodrigues):
rod = Vector((rodrigues[0], rodrigues[1], rodrigues[2]))
angle_rad = rod.length
axis = rod.normalized()
if armature.pose.bones[bone_name].rotation_mode != 'QUATERNION':
armature.pose.bones[bone_name].rotation_mode = 'QUATERNION'
quat = Quaternion(axis, angle_rad)
armature.pose.bones[bone_name].rotation_quaternion = quat
return
def export_animated_mesh(output_path,armature,mesh):
# Create output directory if needed
output_dir = os.path.dirname(output_path)
if not os.path.isdir(output_dir):
os.makedirs(output_dir, exist_ok=True)
# Select only skinned mesh and rig
bpy.ops.object.select_all(action='DESELECT')
armature.select_set(True)
mesh.select_set(True)
if output_path.endswith('.glb'):
print('Exporting to glTF binary (.glb)')
# Currently exporting without shape/pose shapes for smaller file sizes
bpy.ops.export_scene.gltf(filepath=output_path, export_format='GLB', export_selected=True, export_morph=False)
elif output_path.endswith('.fbx'):
print('Exporting to FBX binary (.fbx)')
bpy.ops.export_scene.fbx(filepath=output_path, use_selection=True, add_leaf_bones=False)
elif output_path.endswith('.bvh'):
print('Exporting to BVH (.bvh)')
bpy.ops.export_anim.bvh(filepath=output_path, check_existing=True, filter_glob='*.bvh', global_scale=1.0, rotate_mode='NATIVE', root_transform_only=False)
else:
print('ERROR: Unsupported export format: ' + output_path)
sys.exit(1)
return
if __name__ == '__main__':
try:
if bpy.app.background:
parser = ArgumentParserForBlender()
parser.add_argument('--input', dest='input_dir', type=str, required=True,
help='Input directory')
parser.add_argument('--output', dest='output_path', type=str, required=True,
help='Output file or directory')
parser.add_argument('--pose', type=str, default=None,
help='Always use specified gender')
args = parser.parse_args()
input_dir = args.input_dir
output_path = args.output_path
pose_path = args.pose
if pose_path is None:
exportAnimation = False
else :
exportAnimation = True
obj_path = os.path.join(input_dir,'T-pose.obj')
skeleton_path = os.path.join(input_dir,'skeleton.bvh')
weight_path = os.path.join(input_dir,'weight.npy')
init_scene()
import_obj(obj_path)
mesh = bpy.context.selected_objects[0]
import_skeleton(skeleton_path)
skeleton = bpy.context.selected_objects[0]
mesh.select_set(False)
skeleton.select_set(True)
bpy.context.view_layer.objects.active = skeleton
bpy.ops.object.mode_set(mode='EDIT')
for bone in skeleton.data.edit_bones:
bone.use_connect = False
target_vec = Vector((0,0,-1))
vec = Vector(bone.head - bone.tail)
rot = vec.rotation_difference(target_vec).to_matrix().to_4x4()
R = (Matrix.Translation(bone.head) @
rot @
Matrix.Translation(-bone.head)
)
bone.transform(R)
bpy.ops.object.mode_set(mode='OBJECT')
mesh.select_set(True)
skeleton.select_set(True)
bpy.context.view_layer.objects.active = skeleton
bpy.ops.object.parent_set(type='ARMATURE_AUTO')
weight = np.load(weight_path)
for vg in range(weight.shape[1]):
vg_index = str(vg).zfill(2)
for i in range(weight[:,vg].shape[0]):
if weight[:,vg][i]> 0:
mesh.vertex_groups[vg_index].add([i], weight[:,vg][i], 'REPLACE')
#Set Animation
if exportAnimation:
anim = np.load(pose_path)
poses = anim[:, :-3]
locs = anim[:,-3:]
poses = poses.reshape(poses.shape[0], -1, 3)
for frame in range(poses.shape[0]):
for j in range(poses.shape[1]):
if j==0:
continue
#NOT SURE ABOUT LOCATION
#skeleton.pose.bones['00'].location = locs[frame]
#skeleton.pose.bones['00'].keyframe_insert('location', frame=frame)
bone_name = str(j).zfill(2)
set_pose_from_rodrigues(skeleton, bone_name, poses[frame][j])
skeleton.pose.bones[bone_name].keyframe_insert('rotation_quaternion', frame=frame)
export_animated_mesh(output_path,skeleton,mesh)
except SystemExit as ex:
if ex.code is None:
exit_status = 0
else:
exit_status = ex.code
print('Exiting. Exit status: ' + str(exit_status))
# Only exit to OS when we are not running in Blender GUI
if bpy.app.background:
sys.exit(exit_status)
Still have some issues...
Could you share some scripts that I can generate pose.npy (e.g. ./eval_constant/sequence/dance.npy) It would be helpful to fix some errors ..
Hi, really sorry for my late reply. I was too occupied in the past weeks : (
I updated the code to support output the animated bvh file with the parameter --animated_bvh=1
. Now I think it should work with your code (without exportAnimation section).
And thank you for sharing your amazing code with everyone! You are welcome to start a pull request to add your code as part of the repo : )
Let me know if there is any other issue.
Thanks to animated bvh , I could solve the issues easily. Simply I've just used animated bvh to rig , instead of calculating rotation of joints which cause many issues due to the rigging system in blender.
Furthermore , I've add some functions to generate neural-blendshapes model. (Only envelope-only model was generated from my previous script)
To do so, "coff.npy" and "basis.npy" should be exported first. I add the code below in the forward function of ./architecture/blend_shapes.py
np_basis = basis_full.detach().cpu().numpy()
np_coff = coff.detach().cpu().numpy()
np.save('./demo/basis.npy',np_basis)
np.save('./demo/coff.npy',np_coff)
Result (Comparison between obj seq and fbx)
I've requested a pull as well. (nbs_fbx_output.py)
Closing this issue as it's solved already
Thanks again for your fantastic code! I've merged your pull request and updated the code for saving blend shapes stuff and README.md.
First, thanks for your great work.
Just wondering if exporting animated format(such as bvh or fbx) is supported ? Looks like demo.py only exports T-pose skeleton . I've tried to combine all outputs to fbx format but had trouble with animating character with a given pose. (In blender) I used pose (FrameN, Joint_Num,3) data as an input of euler value but it doesn't work.
Any Suggestions??