[TRACKER] Port to C++ for Godot Engine

[fire]

I'm trying to scope out what needs to be ported for the C++ game engine Godot Engine.

• Port C#? Java to C++
• Make a visualizer for kusudama constraints

The goal is to have realistic IK tracking given three sensors (head-mounted display, two trackers) and at least 3 - 10 VR tracker sensor positions.

My peers had some concerns about FABRIK's unnaturalness around the bending of the elbows, given only three sensor points.

The plan currently is to have analytic solvers for the IK system, but it doesn't have sound theory behind it.

5 mentioned some calibration needed for scaling the body. We could use a "T" or "A" pose for doing that. Other systems like VRCHAT or HI FIDELITY used the height of the head-mounted display and the hips to give a possible scale. Scaled avatars like huge persons or small persons are a problem.

[EGjoni]

Why does the Godot IK api split the rotation in euler angles from the translation when the Transform data structure handles everything.

Maybe a user convenience? (Easier for humans to think in Euler angles).

Godot's internal Transform is a 3x4 matrix consisting of a 3x3 matrix for Basis (rotation and scale) and an Vector3 origin for translation.

This is a highly atypical representation. Usually transform matrices are square matrices. In 3D, a 4x4 matrix is used with the fourth column or row representing perspective.

My thought was to remove the current_ori and current_pos and use one transform. Do I want it to be global or local? Do I want a transform for the global transform (relative to the root bone) and a local transform (relative to the parent bone).

For each Bone, you want a single transformation defined relative to its parent bone's transformation. The bone's global transformation is important for doing a bunch of things, but that global transformation can and should only ever be inferred by composing it with all of its ancestor transformations. The result of that composition should ideally be cached, so that the global transformation is only ever recomputed when one of its ancestor transformations change, and even more ideally so that the recomputation traversal is carried out only up to the ancestor that was changed.

That entire process (of defining transformations relative to parent transformations, caching the global transformation that results from composing that transformation with all of its ancestors, and keeping track of whether a global transformation should be recomputed before returning the result to the user), is what the AbstractAxes class is dedicated to.

I imagine the Godot engine must have SOME scheme that does something similar with its own transformations internally (or else the Godot engine is highly inefficient).

A Basis is convertable to Quat.

Almost. A Basis also contains a translation component. So, a Basis is convertible to a Quat + a Vector3.

My use case was to demonstrate your proof of concept to interested friends without learning git, processing IDE and other misc thing. I wanted other people to evaluate the proof of concept. Someone who's an artist or a rigger.

I see. Yes that's probably a good thing for the library to have in general. Processing is pretty finicky about directory structures, so if you haven't managed to get self contained executables working, I'll make some and link you hopefully sometime tonight. If you HAVE managed to get them working, then feel free to contribute them back ;).

[fire]

https://github.com/fire/Everything-Will-Be-IK-Processing/tree/v3-rc Is my branch with the fixes to

• ewbIK/examples/FullExample/SinglePrecision/SinglePrecision.pde
• ewbIK/examples/FullExample/SinglePrecision/UI.pde
• ewbIK/examples/HumanoidLoaderExample/SinglePrecision/Human_Holding_Item/Human_Holding_Item.pde
• ewbIK/examples/HumanoidLoaderExample/SinglePrecision/Human_Holding_Item/UI.pde

I have also made builds here:

• https://drive.google.com/open?id=1LYIquZi1Uj9i7dG-x7KpzLAQsKVz0FOG

[fire]

Most 4x4 matrixes can be converted to Godot's 3x4 matrix as I discovered.

The Godot::Basis is only rotation and [scale].

For the IK system I've used the ChainItem as a scratchpad for in progress IK changes that are a duplicate of the existing Godot::Skeleton (with bones) system. So if I go to the pure local transform schema, I'd need to add caching.

[Edited]

Chain is Godot:Skeleton, while ChainItem is a bone equivalent.

There's another data structure for a camera.

[EGjoni]

I have also made builds here:

Great, thanks!

Most 4x4 matrixes can be converted to Godot's 3x4 matrix as I discovered.

Yes, it's not difficult. It's just a weird choice.

So if I go to the pure local transform schema, I'd need to add caching.

Well, if it's just a scratchpad, you can just have a getGlobalBasis() function that recomputes the global transform each time you need it, and worry about optimizing later. Alternatively, you can just start with a naive port of AbstractAxes. Ultimately, the IK Solver works by creating and maintaining a separate internal representation of the skeleton anyway (so intermediate computations don't interfere with what's being displayed, and so that results are atomic), so converting between the two representations on the mimicry step wouldn't be much harder than mimicking under the same representation.

But I would be fairly surprised if Godot doesn't already have some caching scheme implemented. If it doesn't, you might want to consider taking this opportunity to give it one.

I can take a look at what Godot is doing if you wanna point me in the right general direction.

[EGjoni]

Alright, I looked into Godot's scheme a bit. I believe their Spatial class is a very close analog to ewbIK's AbstractAxes class. And Godot's spatial class does indeed seem to take care of caching and recomputing the global transformation as needed.

Godot's Transform class analogizes most closely to ewbIKs AbstractBasis class, And Godot's Basis class analogizes most closely to ewbIKs Rot class.

Let me know if you need help.

[fire]

Note that Spatial is a scene node, while Transform and Basis are objects.

[EGjoni]

Does it being a scene node present some implementation problem?

[fire]

Yes, because I shouldn't need to add new spatial nodes. The bones is in the skeleton node.

The skeleton ik cmdd is a different node.

Currently I'm trying to discover what is global, what is skeleton global and what is bone local. This explains why the bones are [at] the root of the mesh rather than relative to the other bones.

Edited:

There's also root of the chain vs a bone in the chain. NOTE: This isn't the root bone of the skeleton.

[fire]

Here's a comparison. Note the location of the skeleton root. (the orange bones start from a point near the feet).

[EGjoni]

(the orange bones start from a point near the feet)

Are you sure? Unless you're using an atypical visualization of Bones, it looks rather like your bones start at the hands and end at the feet.

Currently I'm trying to discover what is global, what is skeleton global and what is bone local. This explains why the bones are [at] the root of the mesh rather than relative to the other bones.

For reference:

• Bones are transformed relative to their parent bone. The algorithm assumes this.
• Kusudamas are transformed relative to the parent bone of the bone they are constraining. The algorithm assumes this.
• The simLocalAxes corresponding to any bone are transformed relative to the simLocalAxes corresponding to their bone's parent bone. The algorithm assumes this.
• The simConstraintAxes corresponding to any bone are transformed relative to the simLocalAxes corresponding to their bone's parent bone. The algorithm assumes this.
• IKPins are, if I remember correctly (but I'll double-check) transformed "global-ish", in that by default their transformations are defined relative to the skeleton root's parent. If the skeleton's parent is the global transformation, then IKPins are defined relative to global coordinates. If the skeleton root's parent is defined relative to a moving bus instead, then placing an IKPin for the skeleton's wrists on a handrail inside the bus will cause the skeleton to always keep its wrist at that spot on the handrail, even if the rest of the skeleton is scooting over to the next seat. But this is just the default. You are free to redefine IKPins as being relative to basically anything at any time. In the humanoid box example, I redefined the IKPin transformations of the humanoid's wrist as being relative to the transformation of the box.

(If I didn't make IKPins default transformation be relative to the skeleton's parent, then you should definitely make it like that for your port!)

[fire]

I noticed something architecturally different.

Thank you for you explanations.

IKPins are, if I remember correctly (but I'll double-check) transformed "global-ish",

The Godot IK API uses has a (current) limit of a single goal that is in the world. In this demo it is a button on the wall.

This project appears to add ik pins all the way to the root.

I'm calling pins Godot ChainTargets.

Here is the current code. The c++ cpp is the implementation in the same folder.

https://github.com/fire/godot/blob/ik-cmdd-kusudama/scene/animation/skeleton_ik_cmdd.h#L631

  public void updatePinList() {
    pins.clear();
    recursivelyAddToPinnedList(pins, humanArmature.getRootBone());
    if(pins .size() > 0) {
      activePin = pins.get(pins.size()-1);
    } 
  }

  public void recursivelyAddToPinnedList(ArrayList<IKPin> pins, Bone descendedFrom) {
    ArrayList<Bone> pinnedChildren = (ArrayList<Bone>) descendedFrom.getMostImmediatelyPinnedDescendants(); 
    for(Bone b : pinnedChildren) {
      pins.add((IKPin)b.getIKPin());
      b.getIKPin().getAxes().setParent(worldAxes);
    }
    for(Bone b : pinnedChildren) {
      ArrayList<Bone> children = b.getChildren(); 
      for(Bone b2 : children) {
        recursivelyAddToPinnedList(pins, b2);
      }
    }
  }

I wonder if this is a significant change?

It is possible to add all the bones from the root (which root? chain root, or skeleton root) to the target array.

I'm going to not support multiroots in Godot yet (Godot has support for multiroots).

[fire]

I'll post the ik demo here so I can reference it.

Note that it has only twist constraints and zero direction constraints.

ik.zip

[Edited]

Good night!

[EGjoni]

This project appears to add ik pins all the way to the root.

I'm not sure what you mean. ewbIK only adds IKpins on the bones the user wants to pin.

The Processing code you quoted is just for UI purposes. It traverses the skeleton to find which bones have pins attached, so that the user can cycle through them with the up and down arrows.

It is possible to add all the bones from the root (which root? chain root, or skeleton root) to the target array.

Within the actual library itself targets aren't kept track of in an array, they're kept track of in a tree structure represented by the SegmentedArmature class.

I can (and probably should) draw a diagram if you feel unclear about what's going on there.

[fire]

Ah, I'm stuck trying to find why the bones aren't attached to their previous bone and why the position is at the floor beside the feet.

Thanks for your help though. Holidays is almost over, so things will slow down.

[EGjoni]

At which point in your code does a bones' local transform get composed with the transforms of its ancestors in order to determine the bone's global position/orientation?

[fire]

I'm having a hard time finding the bug. Is there an order of operations than I can verify step by step?

The idea is to start from a good point.

I found several places that grabbed the skeleton's global pose when given a bone id.

[EGjoni]

Oh, sorry. I missed this comment somehow.

We should probably try to discuss this over live chat or something. You can walk me through the approach you're taking in Godot, and I can walk you through what the current library expects.

You can e-mail me with your preferred chat venue at rufsketch1@gmail.com.