Papyrus Inverse Kinematic

[Haeretical]

Would it be possible to add IK animation overrides to Papyrus? This would open up a whole realm of possibilities for animations. Especially object interactions, like having the actor actually picking up objects with the arm reaching for it. I know this is probably a long shot, but NAF already does a few rather advanced things and IK is at least in the animation sector, so if its possible I guess this might be the right place to ask for it. At least theoretically ingame IK should be possible and probably can be found somewhere in the Fallout source, since there are mods that implement it like FRIK for VR. I apologize If this request is out of scope, in which case this issue can be removed of course. In any case thanks for the great work on NAF.

[Deweh]

I've actually been working on exactly that the last few days. Unfortunately, I can't get it working properly. The IK chain correctly points at the target location when outside of reaching range, but when within reaching range the rotations go all out of whack.

It may have something to do with how the CE handles local rotations, or something to do with the weirdness on the Y-axis that I observed when working with euler angles. (When the rotation on the Y-axis is < -90 or > 90, the directionality of the axis inverts.)

[dlinny-lag]

I don't know what API do you use to deal with euler angles, but I know that CE uses quaternions. Maybe this can give you some idea.

[Haeretical]

Oh wow that's better news than I ever would have hoped for. I had to fight with the whole euler to quaternions and inverted rotation issue of euler myself a good while back. To be honest I happily forgot most of it again and I am not that great at coding in general, but if for some reason this can't be solved I am happy to give that part a try myself. Same goes if you need somebody for testing any of this. The function dlinny-lag posted does look like it goes in the right direction though.

[Deweh]

I don't know what API do you use to deal with euler angles, but I know that CE uses quaternions. Maybe this can give you some idea.

@dlinny-lag I am using the engine's built-in functions to convert between rotation matrices/quaternions/euler angles. At the core, the engine uses a 3x4 rotation matrix to represent a node's rotation, (scale is also encoded in this matrix, but that's its own problem), so ultimately a correct rotation matrix needs to be produced for IK to function as expected.

Euler angles are only utilized when manually editing bone rotation via a 3-dimension gizmo. I solved the Y-axis weirdness by switching the euler angle order to ZXY when adjusting the Y-axis, and XYZ when adjusting the X or Z axis.

When solving for IK, the nodes' rotation matrices are converted to quaternions, then those quaternions are passed to the FABRIK solver, then the result is converted back to rotation matrices. So, euler angles are never utilized for IK, aside from any conversions the FABRIK solver may do under the hood on its own. For reference, this is the implementation of the FABRIK solver I'm working with: https://github.com/TheComet/ik

I assume the most likely issue may be a difference in the way the CE and the linked library handle local rotations/positions, since the local position of a node never changes in the CE, despite its current rotation.

[dlinny-lag]

I solved the Y-axis weirdness by switching the euler angle order to ZXY when adjusting the Y-axis, and XYZ when adjusting the X or Z axis.

The function dlinny-lag posted does look like it goes in the right direction though.

These two are related.

Axises in CE are a bit different than it would be expected. I didn't try to obtain full information about axises order and directions, forward vector calculation was enough for me.

[Deweh]

@Haeretical @dlinny-lag I got IK working properly. It's a bit jittery atm since it's using transforms from the previous frame for its calculations, and also the solver can lose track of which way the joint is meant to bend if it's fully extended, but those things can be worked on.

This demo is showing leg IK. This can be trivially transferred to arm IK (or any arbitrary chain of nodes), but I need to fix the issue of it using previous frame transforms first. The F4 human skeleton has three "twist" bones between the elbow and the wrist, so in order for the solver to calculate the correct global rotation for the wrist, it must know the current transforms.

(Video Removed, updated demo posted below)

[dlinny-lag]

It's a bit jittery

I'm afraid that "previous frame tranform matrix" is not the only problem. As I can see it oscillate around some "point". I faced such kind of problem when matrices arithmetic operations was made with single precision numbers. On the large distances (more than 1000 from zero) accomulated error becomes visible due to insufficient precision. I don't know how CE handles this problem.

[Deweh]

It's a bit jittery

I'm afraid that "previous frame tranform matrix" is not the only problem. As I can see it oscillate around some "point". I faced such kind of problem when matrices arithmetic operations was made with single precision numbers. On the large distances (more than 1000 from zero) accomulated error becomes visible due to insufficient precision. I don't know how CE handles this problem.

It's possible there's some coordinate imprecision since the calculations are taking place in world-space. In Diamond City in particular 4 of the 7 digits of precision are being taken up by the whole number, so only 3 digits are left after the decimal point. It's possible to make the IK solver use double-precision, but FO4 uses single-precision, so the end result would still be susceptible to precision issues.

FABRIK is also an iterative algorithm, so the solution may not always be exactly the same. The library includes a specialized single-pass two bone solver, but that seemingly only outputs positions, not rotations. And for skinned IK, rotation is what's really needed.

Either way, I think the solution is good enough for posing and such. I also resolved most of the jitter by doing a double-pass on the IK. In pseudocode:

CalculateIK()
UpdateSceneGraph()
CalculateIK()

For some reason, not doing that first CalculateIK() completely breaks the rotations. It's certainly not great for optimization to do a double-pass, but the algorithm is fast enough that it has no noticable impact on FPS. For configurability, I included a boolean to switch between single-pass (which uses previous frame's transforms) and double-pass (which uses updated current frame's transforms). The only thing the IK solver really needs at this point is constraints, because it completely loses orientation once bones become fully extended.

[Deweh]

After much work on this subject, I've fixed all the jitter & implemented pole constraints to control which way elbows/knees point. I've also optimized the CE->IK->CE flow so that it makes no impact on FPS with 4 active chains.

The IK system will be included in the body animation update.

[Haeretical]

Incredible! Quite excited to tinker around with that.

[Deweh]

Implemented in bb701ae