Handle actions based on animation elapsed (current frame)

So I once again have a problem regarding animations. I want to enable/disable hitboxes based on where we are in the animation (which frame essentially). Say we have an animation with 10 frames and I want to enable a hitbox on frame 3 and disable it on frame 7.

Using trickfilm, our only option right now is to get the elapsed time and also get the clip duration (which you need to do in a bit more complex way) and calculate yourself which frame is currently playing. Though you wouldn't necessarily need the frame, you could also just use timestamps. So you would check the elapsed time and just check your timestamps that you store somewhere separately and handle stuff based on that.

I just wanted to get your view on this, do you have any better ideas of how to go about implementing such a thing? I believe godot has events (or signals) that you can trigger in the animation itself. I really wouldn't want to use any Events when it comes to animation as that is just prone for frame-delays which can result in some really ugly visual glitches when dealing with animations.

Curious if you were thinking of creating some interface that would allow for something like this or if you think it's better to just let the user implement it on their side.

I have been thinking about some asset/reflect based StateMachine/AnimationGraph, but thinking about it is all I have done. Not really relevant for this, though.

Thanks for bringing up a new use case. Definitely sounds like something that should be supported. If you have any additional details of what you need/are trying to do, feel free to let me know. Right now I am imagining how fighting games tie hitbox and animation to "frames". Side note: I feel like a lot of gamedev articles online advocate for decoupling logic from "what you see". At least for this case there is no way around it, but maybe there is not much to gain here anyways.

Regarding exposing current frame index: We should just do that :D

As far as Events go, I did not go very deep, because it seems like the obvious solutions are not very good.

Events via EventReader/Writer are potentially delayed as you said. I do not like the implications of that.
Observers are "instant" (buffered like commands, as far as I understood), but they come with some kind of performance drawback (?); maybe that would be fine, but the most pressing issue is something else
Any event based solutions seems to be way too broad; You do not want events for every frame increment of every animation. And even if we can prefilter that via some flag, how are you supposed to cleanly differentiate your behaviour based on the animation?

When I mentioned AnimationGraph at the very beginning, I was picturing reflected functions as transition conditions (and possibly as state actions). Maybe this right here is a better start to get my feet wet with function reflection (I am not even sure if function reflection is the only/best way to do this):

An AnimationClip (the asset) contains a vec of functions
functions with the following signature: (Entity, &mut Commands, AnimationPlayer2D, AnimationPlayer2D)
- First AnimationPlayer is before current update, other one is after (so arbitrary changes can be detected)
- Not sure if &mut Commands + observer based events or &mut World is the way to go
- Might also make sense to allow for more args to be passed through the asset (to reuse functionality)
Somewhere in the animations update system we call every user defined functions
That way you could have functionality unique to that animation, where you check for frame 3/7 to enable/disable the hitbox (via observer based event/mut world and query on Entity)

But yeah, that is just an idea. Looking forward to get your opinion on this.

If you have any additional details of what you need/are trying to do, feel free to let me know.

Yeah maybe it's helpful if I go into more detail. There are a couple issues I have encountered when trying to get just a simple player statemachine + animations to work. One issue is that I am using just_finished to check if an animation has finished and then to transition into a new state (say the player has finished a punch, go back to idle).

The problem with this is that I have all my state logic run before any trickfilm updates are run. This means that when I have multiple state changes per frame (which I reckon isn't good in the first place, but statemachines and the like are super complicated for me so I am just figuring them out as I go) this breaks because the animation is still saying it's finished when in reality it isn't.

Or another problem: Player just finished attacking, we switch to the recovering animation, then on the same frame we get the input to attack again so we go back to attacking, but because all this happens within the same frame and all of it happens before any trickfilm updates, to trickfilm it looks like you never changed anything, so it just keeps playing the attack animation (and because I use just_finished you are now stuck in the attacking animation).

It's very likely that my implementation is just really bad, but I don't know how you could decouple the animation from the state, at least not without some big drawbacks. My current idea was to just update the animation manuelly whenever you change your state using custom commands, though I haven't implemented it yet. As far as I understand trickfilm, this should reset the AnimationPlayer2D clip so that just_finished would be false.

Side note: I feel like a lot of gamedev articles online advocate for decoupling logic from "what you see". At least for this case there is no way around it, but maybe there is not much to gain here anyways.

This is something that has been bugging me as well. Decoupling it is of course nice if you can do it, but how on earth would that work here? Though I also don't consider trickfilm the "what you see", it's just another backend, it just so happens to also update the "what you see".

As far as Events go, I did not go very deep, because it seems like the obvious solutions are not very good.

Yep, very much agree with your points here, I also don't think that event-like systems are the solution here.

When I mentioned AnimationGraph at the very beginning, I was picturing reflected functions as transition conditions (and possibly as state actions). Maybe this right here is a better start to get my feet wet with function reflection (I am not even sure if function reflection is the only/best way to do this):

Gonna be honest, I don't even know what reflected functions are ¯\_(ツ)_/¯. But anyways, I like this solution. The only thing I dislike a little is that you would have animation specific information in your code, rather than having it all in the trickfilm file. Though I also don't know how you could not do that.

Though one problems remains for me, and that is how to make sure that a State (like the PlayerState) and its AnimationPlayer2D are synced. For example, I have set_state and I am thinking about having a custom command that I always fire which updates the animation player by setting the corresponding animation to the state. But that isn't quite nice because then I always need to pass Commands to set_state. Do you have any ideas? If this seems like a strange issue then it's likely because it is. States are somehow super hard for me to work with and understand. Especially because they work so closely with the AnimationPlayer2D and frame delays can cause bugs so you need to be extremely cautious with the scheduling etc.

Did you implement anything like that in one of your projects? Would love to see some examples of this.

If you have any additional details of what you need/are trying to do, feel free to let me know.

Yeah maybe it's helpful if I go into more detail. There are a couple issues I have encountered when trying to get just a simple player statemachine + animations to work. One issue is that I am using just_finished to check if an animation has finished and then to transition into a new state (say the player has finished a punch, go back to idle).

The problem with this is that I have all my state logic run before any trickfilm updates are run. This means that when I have multiple state changes per frame (which I reckon isn't good in the first place, but statemachines and the like are super complicated for me so I am just figuring them out as I go) this breaks because the animation is still saying it's finished when in reality it isn't.

I am no expert either. All I know is very specific to Simulink Stateflow charts :D. But maybe it is helpful if I tell you how they do it there. First of all, state "updates" are done exactly once per "frame" by a single "driver" (essentially there is just a single code part that takes care of it). In that update at most one transition can be taken to go from one state to another. Transitions have a priority in which way they are evaluated - if a condition is true, all remaining transition are not evaluated anymore.
In your case I would have a single system that queries all necessary data and updates the state machine. I suppose you would also need a way to find out if you are "re-entering" an already active state and then trigger start instead of play on AnimationPlayer2D.

Gonna be honest, I don't even know what reflected functions are ¯\_(ツ)_/¯. But anyways, I like this solution. The only thing I dislike a little is that you would have animation specific information in your code, rather than having it all in the trickfilm file. Though I also don't know how you could not do that.

I have not taken a proper look at them either :D. But what I hope to get from them is referencing function by type path in the asset file. Of course the code needs to exist in rust, but if it works the way I hope it does, trickfilm could already provide a good amount of functions (even generic functions, though users would need to monomorphized those themselves).

Though one problems remains for me, and that is how to make sure that a State (like the PlayerState) and its AnimationPlayer2D are synced. For example, I have set_state and I am thinking about having a custom command that I always fire which updates the animation player by setting the corresponding animation to the state. But that isn't quite nice because then I always need to pass Commands to set_state. Do you have any ideas? If this seems like a strange issue then it's likely because it is. States are somehow super hard for me to work with and understand. Especially because they work so closely with the AnimationPlayer2D and frame delays can cause bugs so you need to be extremely cautious with the scheduling etc.

Did you implement anything like that in one of your projects? Would love to see some examples of this.

I am pretty sure you are the only using this crate :D. I initially created trickfilm for a simple topdown horse game and noticed that I would want different kind of horses with the same animation "data", but that never really went anywhere. Anything I have done since, has not used any animation or was 3d. I really need to do something with the crate to see myself what works and what is still missing, but in the mean time you are all the feedback I got. As for the specific problem: I would try to have only one source of state updates, if possible (may not be possible in your case or just not a good solution in general). The idea is if you only have one update, then it is trivial to sync the AnimationPlayer. Another solution may be to queue up state changes and work through them to update the AnimationPlayer.

I am no expert either. All I know is very specific to Simulink Stateflow charts :D. But maybe it is helpful if I tell you how they do it there. First of all, state "updates" are done exactly once per "frame" by a single "driver" (essentially there is just a single code part that takes care of it). In that update at most one transition can be taken to go from one state to another. Transitions have a priority in which way they are evaluated - if a condition is true, all remaining transition are not evaluated anymore.

Haha nice. But that is actually a pretty good idea. This sounds similar to a behaviour tree (in that you also have a priority of evaluation and stop once you reach a leaf node and only have a single updater, though just in a much simpler format).

I have not taken a proper look at them either :D. But what I hope to get from them is referencing function by type path in the asset file. Of course the code needs to exist in rust, but if it works the way I hope it does, trickfilm could already provide a good amount of functions (even generic functions, though users would need to monomorphized those themselves).

Oh that sounds pretty good. That would definitely be very useful to have.

I am pretty sure you are the only using this crate :D. I initially created trickfilm for a simple topdown horse game and noticed that I would want different kind of horses with the same animation "data", but that never really went anywhere. Anything I have done since, has not used any animation or was 3d. I really need to do something with the crate to see myself what works and what is still missing, but in the mean time you are all the feedback I got.

Hahahah, a heavy burden to bear. Well I am glad you ended up creating trickfilm, I really like the whole concept and the API. I also appreciate you putting in the work to update trickfilm despite not actively using it. How is 3D in Bevy btw? I know that 3D is just way more complicated then 2D in general, so I can't quite imagine 3D being super nice to work with in Bevy.

As for the specific problem: I would try to have only one source of state updates, if possible (may not be possible in your case or just not a good solution in general). The idea is if you only have one update, then it is trivial to sync the AnimationPlayer. Another solution may be to queue up state changes and work through them to update the AnimationPlayer.

Yeah now that I am thinking about it, it's actually a pretty good idea. My only problem with this is that I would need to query for everything I need in the system that checks all the transition conditions. It's just not very scalable (to be fair, state machines in the first place aren't super scalable, but this just makes it even worse). I am wondering what a solution could be. In essence, I just want to be able to call functions from that main system but not need to pass all the bevy parameters, so something like a one shot system or whatever, but that would be pretty bad in a system that gets called every update I think.

Perhaps you could just use &World in the main system and then call all the other systems from there that perfrom the transition checks only needing to pass &World. I think that should work?

Hopefully fixed with the event update.

KirmesBude / bevy_trickfilm

Handle actions based on animation elapsed (current frame) #27