Rokoko / rokoko-studio-live-blender

Rokoko Studio Live plugin for Blender
GNU Lesser General Public License v3.0
326 stars 38 forks source link

Rokoko doesn't record keyframe in Quaternions. Hard to convert from Euler. #94

Open B-Skizzle opened 2 years ago

B-Skizzle commented 2 years ago

I noticed that this plugin only records in Euler rotations, which normally wasn't a problem for me until I started using the NLA editor. I've noticed that when I tried to blend various animations, weird effects would occur like this:

image

The middle image shows a weird tweening between clips.

I traced it down and it's mainly because the rotations on the bones are in Euler (which don't rotate in the shortest amount of distance) instead of quaternion. I tried smaller examples and verified that this problem doesn't occur when the bones are in quaternion, because quaternions do rotate in the shortest amount of distance). So, I tried to use Discontinuity (Euler) Filters, but it didn't solve the problem.

I've also tried plugins (Rigify) that try to convert the euler keyframes to quaternion keyframes, but it requires that fcurves belong to a group, which after looking in the code, is left omitted (in recorder.py).

image

I had to modify the script myself to have each fcurve belong to a group with the bone name, then I was finally able to use rigify's conversion tool. It would be nice not to have to do this every time.

Long story short, is it possible to also record quaternion keyframes? It would help prevent a lot of euler rotation problems in the future.

Darkblader24 commented 2 years ago

This is a great suggestion, thank you! I will add this asap.

B-Skizzle commented 2 years ago

I also hate to be bothersome but there's another issue. When the bone data is live captured and recorded, the "inherent rotation" is turned off and recorded as keyframes. This is troublesome in the end because it prevents me from adjusting editing the mocap like at 8:04 - 8:13 in this video:

https://youtu.be/nB5qhTItryU?t=484

In the video, the children bones follow the parent, making it possible to adjust the mocap in the nla editor.

This is not possible using the blender plugin though. For example, this is what it looks like before I make an adjustment to the pelvis bone, which is the parent of all of the other bones:

image

This is what it looks like afterwards: image

You can see that the children bones don't change because of the inherent rotation.

I found myself having to rotate each individual bone along the chain to make an adjustment. I know the work around is to export a recorded .fbx animation from Rokoko and then retarget it to my desired rig, but it will become very tedious for me because of my workflow, which requires that I export tons of animations.

I checked out the python code for the inherent rotation and it looks very complex (to me, at least). How hard would it be to live record without using inherent rotation in a future version?

Darkblader24 commented 2 years ago

Thank you for the suggestion, keep them coming!

Unfortunately, recording without inherit rotation was a compromise, because having it turned on made the animations very jittery and there was no easy solution for it. I will look into this but it will take a while.