LocalToModelJob that outputs transforms rather that matrices

[kylawl]

So we're working on a couple different things right now which (I think) make sense to do with TRS transforms instead of matrices.

• Animation powered physics. We have some model space varlet simulation which needs to be blended with/powered by sampled animation.

• Motion interpolation without sampling animation. We're looking at solidifying our engine to use fixed time steps for all game logic which in some cases includes animation. What this means is that when our skinning system extracts animation poses, we'll need to keep around the previous pose in order to interpolate the skinned model between game updated.

I'm curious what your thoughts are on this feature, is it something that would be valuable or am I missing an obvious or subtle reason why doing everything in local space is just easier or better?

[guillaumeblanc]

I think it can be valuable to compute model-space TRS transforms rather than matrices sometime. The use case I was thinking about was dual quaternion skinning, as dual quaternions would be easier/faster to build from TRS transforms. ozz LocalToModelJob makes a good use of SoA quaternion to matrix conversion though, which I think is quite optimal.

Also I have a doubt about the fact that TRS cannot store all transformations, unlike matrices. I wondering if a mirrored or non-uniform scale in local space could create model space transformation incompatible with TRS. What do you think?

You probably know that already but I think local-space is required for blending. For example in model-space you could not animate the upper and lower body with different animations, because transformation on the lower body need to be propagated to the upper body (during local to model-space update). Right? Also optimisation wise: bones move a lot less in local-space (fingers can remain fix while hand is moving), leading to far better key frame reduction. I think local-space can be more precise when compressed. Again, in local-space bone translations are usually fixed, thus unaffected by compression.

For your "physic" use case, I don't really know. Using local-space data you could convert rotation to torque, and power physic with that. From physic to animation, you can still compute local transforms back. Interpolating previous and next frames should work if keyframes are very close, which should be the case with fixed time steps. I would try to avoid sampling animation during your game logic update though, letting animation sampling/blending for rendering (non fixed time steps). If you need to animate things (like events, motion...), you could use tracks (float 1 to 4, quaternion). They can be sampled independently from the animation indeed (that's the design I opted for), and store in local or model-space at will (it's user data anyway).

I'm not sure it helps, don't hesitate to give more details about a specific example.

[kylawl]

In the case of the non-uniform scale, you're right. We'll end up with weird animation that differs from the source animation. - As an aside, stretchy rigs generally rely on being able to scale a bone without scaling children. Say for example a squash/stretch spine where you don't want the arms, legs, neck and head to also scale. I wonder if that's a feature we want to look at in the future? Like a bitmask for how a joint inherits from it's parent or something?

In our use case, we use a verlet simulation with constraints to determine the start/end point of each joint in model space. We use the positions and the source animation twist to compute reasonable rotations. In the case where we want this to feed back into the animation blending job, it's probably better to simply convert this back into a local pose. As you said, we can't meaningfully blend in model space anyway.

If you need to animate things (like events, motion...), you could use tracks (float 1 to 4, quaternion). They can be sampled independently from the animation indeed (that's the design I opted for), and store in local or model-space at will (it's user data anyway).

This is actually a good insight into how you intended the system to work. Aside from events and extracted motion, other important game play data being baked out as a separate "slim" animation wasn't something I had considered. I just figured I would sample the whole animation and carry on.

I'll just close this issue now, thanks :)

[guillaumeblanc]

I'll keep the LocalToModelJobTRS and single bone scaling feature in mind.

You talk about "slim" animations. Just to be sure, you saw ozz tracks already, right? They're small animation indeed ;). I think TriggeringJob brings a bit more than sampling animations though.

[kylawl]

I did know about the user tracks. However, I only considered them from the perspective of animating camera params or lighting data. It never occurred to me to generate additional gameplay specific tracks.

The take away from this is that I'll probably extend our engine's animation system to include a separate set of gameplay tracks that can be paired with the normal animation. That also makes it easy to generate low footprint server-side packages for running dedicated servers by just stripping the skeletal part.

[kylawl]

Actually, one other note on the inheritance masking. On Sleeping Dogs, we used this heavily to cheaply "retarget" mostly proportionate characters by simply allowing joints to only inherit rotations.