Implement Animation Layering

[controllerface]

Right now, animations may blend between two different states, but this only applies to transitions. In order to improve the visuals of the game, it will be required to allow different parts of the character to have different animations applied to them. For example, the player can currently punch, which causes the animation for punch to be selected. If the player then walks to the left or right while holding the button for punching, their legs do not enter the walking animation. In essence, you can't punch wand walk at the same time, at least visually. You do move while punching, but the animation is not correct.

In order to make this work, some buffers will need to be updated. I'm not sure exactly how I'm going to implement it yet, as there's a few things to consider. Ideally, I feel like each hull or hull bone should have some value that determines what layer it is "in" for the purpose of being animated, and also the entity needs to have multiple "slots" that denote what animations are currently active.

I need to do some more research on how this works in some common game engines and see if I can get a feel for how it works under the hood. I remember looking into this a bit a while back, and I recall some insight in that layers can be implemented very similarly to blending, and since I have logic for that already, I would like to try and leverage it. Conceptually, it makes sense to me that a bone in layer A, which is currently set to animation X, and a bone in Layer B, set to animation Y, could have a blend factor for A and B, both set to 100%, which would cause their respective animations to apply totally. The mechanism for this still is fuzzy to me.

[controllerface]

After a somewhat less that fruitful detour into CCD, picking this back up. I think I have some ideas how to go about this now.

Looking at existing games engines for inspiration was a bit less useful than I'd hoped, but not unexpected. I don't want to build in a complex workflow for layering animations, and I also don't need to do a crazy amount of layers, I think I can even just start with 2, an upper and lower body "slot". I might make the lower body the "default" and then make the upper body a layer that certain bones can "opt in" to. That way, I can have some base layer and then any object that cares about a different layer can "choose" to be affected by that layer or not. If the layer is unset, then it can default to the base. As long as there is always a base, I think it should work.

If this goes well, I could add more layers, and i can align them with the cl vector sizes. So if I wanted to, that would give me up to 16 layers if I really wanted to go nuts, which I think is probably more than enough.

[controllerface]

After thinking and then re-thinking, and then doing some sketches.. I think I've thunked upon an approach i can move forward with:

• I will partition the current animation index buffer into 4 layers

  • x=0: the base/idle animation layer
  • y=1: lower body, walking/running/etc.
  • z=2: upper body, punching/interacting
  • w=3: head/neck, unused for now, but will add "look-at" later

• all animations will be assigned to a layer. When an animation is entered, the index is simply put into the matching layer position

• all bones will be designated a "layer level". When bone animations are selected, the highest layer with an animation set is chosen

  • layer 0 will always be set to ensure there is always some animation to pull from
  • allows upper body to have animation data for locomotive actions that can be overridden by player punching and similar actions

This is the general idea, and there are a few things that need to change in order to make it easier to pull this off. The most important one is probably moving the current logic out of a kernel and onto the host. I am quickly seeing how the logic is going to need to get a lot more complex and as this is something that only happens to one entity per frame (i.e. the player) there is truly no benefit to doing it on the GPU, it just makes it more cumbersome. This means I need to have a point where some player data is read back from the GPU, but I already have a point where that occurs or the camera positioning data, so I should be able to simply read back more data in that spot, rather than add several more individual reads, which would be less performant.

Once this is done, I then need to work on decoupling the player's "action" state from their animation state. Right now, the logic is hard coded to key off of whatever animation is currently active, but I need to have multiple active states now, one for each layer. State transition logic will need to be run differently that it is now, and I don't know exactly what that will look like yet, but I do know it will require separate bits of logic for different sections of the body.

So, not unexpected but a hefty chunk of work is ahead. Let's see how it goes 👍

[controllerface]

Finished pulling the state change logic from the GPU kernel into the Java side, and took some very cursory steps to slim things down a bit. Without the CPU/GPU barrier there's some stuff that is extraneous now, like the control points enum values, which basically just mirrors the input mapping enum, and the input/output state structs. These can be removed or trimmed down drastically. Hoping to have some time tomorrow to actually start implementing the layering mechanism. I am mostly focused on how blending will work, since that will be a little more complex with 4 different states to be coming from.

[controllerface]

Did a quick "tidying up" to the new code and removal of the old GPU based code for player movement, should set me up for a clean slate when I start working on the layering.

I took a little time to try and remove any extra object creations that happen during the state logic. This isn't something I want to get too obsessed with, but I do want to make sure I'm not creating objects where it isn't necessary. The state change code is all single threaded, and is called only once per tick, so all the objects it uses can be created once and reused, so I did that. There's a couple other places I could start making changes like this in the future, but won't rabbit-hole on it for now.

[controllerface]

Got a lot of the "boring" work done, new buffer for bone layers created and wired up, added a new enum to determine layer affinity for bones. I am keeping the layers to just 2 until i get everything working and then will expand it out to 4 as intended. The idea is to use layer 1 for the upper body bones, and 0 for everything else. I have the kernel ready where it can determine layer preference for each bone, a=so now the fun part of actually getting the layers selected.

I am finding the hard part is getting my head around how state transitions should work now. With just a single layer, the logic is easy, since you are going from one state to another. Now though, body sections are moving between states independently, so the existing logic will need to be modified.

My first though was to try and leave the state transition switch block as-is, and just "route" the state to the proper layer, but that starts to get a little bit weird for transitions. I am leaning now toward updating all of the state change logic to handle all 4 states at a time, but that will make those blocks a bit more complex, as every branch needs to know about each layer. And also, the switch itself kind of breaks down because it expects a single value to switch on...

I will have to ponder this a bit. I'm sure there's a solution.

[controllerface]

OK, so I think the hey here might just be that i need to have a state for each layer, and in fact this is going to become a core part of the character class(es).

1. Layer one will be controlled by the "Idle" state, which can just have the standard standing pose for now, I can add a second idle animation later, probably for being int he water, like a floating animation. I envision this almost like a "passive" state, that is selected based on the environment.
2. Layer two will be the "locomotion" state. This state is affected by movement, i.e. waling left and right, swimming up and down, jumping, falling, etc. All of these states will go into this group. When there is no action, this layer becomes unset, so bones that use it will fall back to layer 0
3. Layer three will be the "action" state. This will be pertain only to actions taken by the character's upper body. Right now, that is only the punch animation, but it would make sense to add activation, door open, and other actions to this later
4. Layer four will be the "look" state, but as I mentioned I will leave this as a todo for later, I want to keep this as focused as possible.

So now, when I run my state transition checks, I need to do all 4 of the state transitions, and then set the animation values accordingly. This should allow me to keep the transition timing map that I have now, but will mean I remove the current animation phase transition code, as the new states will supersede it.

[controllerface]

It's not perfect, but it kind of works! The blending is a little messed up, which is not surprising, I need to expand the blend buffer so it has enough data for 4 layers, right now it only has one and I am using it for both.

But, I can walk or run forward and punch at the same time, and the arms animate separately. Pretty happy with the way this came out, I ended up just mapping the various new player states to the existing animation states which was pretty easy to do. I think these 3 categories (idle, movement, action) are a solid choice, now that I see everything laid out.

I will have to sit down and figure out how each bone should blend to the next animation, will probably need to do some sketches. I can kinda see a couple ways this could go, but nothing super clear in my head yet.

Will sleep on it 🛌

[controllerface]

While still not perfect, it's looking a lot better than yesterday. I moved a little more of the logic into the CPU side, and I have added the "fallback" mechanism for layers. It does do what I intended in that bones from layer 3 fallback to layer 2, layer 2 to layer 1, etc. But the blending is not quite correct in all cases.

It looks like when there's a good amount of time between transitions, they work correctly, but a move to one animation and then immediately to another can cause jerkiness. TBH, I think this was probably present before but it was harder to notice because all the hulls moved together, and I could blame some of the jank on the framerate.

I will continue to look for examples or ideas on how to implement the layout I want. I will also continue to poke at it, because I may just stumble upon an answer. It already looks decent enough too, so I am not the least bit disappointed with the current state. You can legitimately use your upper body and lower body as separate entities and they will animate accordingly. I will need to add more action and idle animations for sure, just to see how things look.

[controllerface]

Animation is now looking quite a bit better, it's still got some hitches, it's difficult to cover every case without introducing crazy animation artifacts. Slowly but surely though, will hash it out. Seeing the character able to attack and move at the same time it pretty great, it's still far from a polished, but it certainly looks more game-like, which at the end of the day, is the goal.

I will probably continue to poke at this for a bit, but if I find I can't quite get every single animation case to blend smoothly, I will likely close this and move on to the next thing. It already is tons better than it was.

One thing I will note though, as expected, there's a small FPS cost to doing things CPU side, that get's a bit worse the more data you read back. It's not tanking perf completely, but it's noticeable and the profiler shows clearly that the call to read the player data is pulling the physics thread down a hair. Considering the complexity that was required to make this even function, I'm just going to call this the cost of doing business for now. Perhaps when things are more battle-tested someday, I might consider migrating back to the GPU side, but not today.

[controllerface]

One more small step, layer 3 animations now can blend to the current layer 2 animation when layer is idle. There's still some popping when starting a run after a punch has completed, and at the very end of a punch cycle that occurs during a walk cycle.

slow but steady as usual, lol

I will need to extrapolate this logic down to all layers once I nail it down, and will also need to actually use layer 4 in some way to really test things. My original idea was to use this for facial animation, but my test model doesn't have eyes or anything, so I could put the head/neck hulls in that slot and just make some animations where the player looks around, perhaps look up and down. Actually.. I could get crazy and implement a sonic the hedgehog style loop up/down camera pan 🦔 heh. I won't hold myself to that,

So I think there's still work to be done on this, so will leave it in the backlog instead of closing it, but probably going to lean more into the jumping logic and overall character movement stuff for a bit.