c-he / NeMF

[NeurIPS 2022] Official implementation of "NeMF: Neural Motion Fields for Kinematic Animation"
MIT License
156 stars 9 forks source link

save motion to bvh #7

Closed Grem-Lin closed 1 year ago

Grem-Lin commented 1 year ago

Hi! When we save motion data back to bvh format, why do we need order[::-1] rather than using order itself? https://github.com/c-he/NeMF/blob/146a1eade5dd7eb77db8380c7f03adf99bfb09a2/src/holden/BVH.py#L418

Another question: euler() function cannot take in order with value 'zxy', as shown in code: https://github.com/c-he/NeMF/blob/146a1eade5dd7eb77db8380c7f03adf99bfb09a2/src/holden/Quaternions.py#L226 It only takes 'xyz' and 'yzx' and conver quaternion to euler angle. I'd like to have order 'zxy' which is the same in dog bvh file work here. Do you know how to get conversion from quaternion to euler angle with order 'zxy' ? The first link on L249 doesn't work and the second link L 280 didn't have such info.

Thanks!

c-he commented 1 year ago

Since these BVH processing code is taken from Daniel Holden's work, I'm not entirely sure why they take order[::-1] instead of order itself. However, since it worked for me when exporting bvh files for the dog motion, could I know what the exact problem you have if you just export them in the default order?

Grem-Lin commented 1 year ago

I see. Yes, my issue is that the reconstructed dog motion looks very wired via both directly input into blender and loading using the visualization code. The GT motion is pretty close to the reconstructed one, so I feel maybe there are some issues related to bvh load/saving or order. I did some study and search, and now I feel the bvh related code should be correct...

The motion export code looks like this:

# save motion in bvh for visualization
            rotations = matrix_to_quaternion(local_rotmat) # (B, T, J, 4)
            rotations_gt = matrix_to_quaternion(local_rotmat_gt) 
            export_bvh_animation(rotations, trans_gt, args.animal.offsets, args.animal.parents, output_dir, f'recons_{offset}', args.animal.joint_names, args.data.fps)
            export_bvh_animation(rotations_gt, trans_gt, args.animal.offsets, args.animal.parents, output_dir, f'recons_gt_{offset}', args.animal.joint_names, args.data.fps)

Do you have any ideas of why I got wired motion? I attached the motions. Thanks! motion.zip

c-he commented 1 year ago

I saw you used export_bvh_animation, did you comment out align_joints in: https://github.com/c-he/NeMF/blob/146a1eade5dd7eb77db8380c7f03adf99bfb09a2/src/utils.py#L229 This function is only used for SMPL cases, so for dog motions you won't need it and should replace it with rotation = rotations[i].

Grem-Lin commented 1 year ago

Oh... I didn't comment out this. I don't know that this is only for human motion. Commenting out this works on my side. Thank you!!