Tracking Issue: MVP Animation

[james7132]

This issue is meant to be a more actionable/scoped version of #91 and #95, specifically scoped to the goal of providing a minimum low level API for managing asset based animations in Bevy.

RFCs

The overall design for these systems are detailed in the following RFCs. Later parts of the implementation are blocked on the design feedback on these individual RFCs.

• [ ] bevyengine/rfcs#49: Low level storage ans sampling design.
• [x] bevyengine/rfcs#51: Composition API and sampling strategy

Implementation/Merge Strategy

This is a fairly large area that touches multiple core areas in the engine. Even with a complete prototype, introducing the system into Bevy in one large PR is likely difficult or impossible (see #1429). The following individual segments will likely need to be individually addressed to piecemeal introduce parts of the system into Bevy proper.

• [x] Assets: #3751
• [x] Assets; #3722
• [x] Rendering: #4238
• [x] Rendering: Rendering Morph Target Displacements
• [ ] Storage: Curve traits and basic concrete implementation (CurveFixed)
• [ ] Storage: Curve compression/quantization
• [x] Storage: AnimationClip Asset
• [ ] Storage: AnimationClip Serialization
• [x] Sampling: Basic skeletal animation player supporting start, stop, seek operations.
• [x] Composition: Basic AnimationGraph implementation.
• [x] Application: Skeletal animation application systems
• [ ] Application: General Reflect-based animation application systems

[cart]

A full property based animation system is largely enabled by the designs in the RFCs listed; however, at the time or writing, there currently exists no way to get references to components that implement Reflect via the component name. This blocks the ability to apply sampled and blended values to arbitrary component properties.

This is already possible. We use this functionality for our scene system. TypeRegistry::get_with_name gets you the type registration, which can then be used to get the ReflectComponent impl, which will return you an Option<&dyn Reflect> (or mutable variant) for a given entity.

[james7132]

@cart: updated the main issue to reflect (haha) this. I'll need to dive deeper into the actual reflection APIs to devise an implementation strategy though.

[Type1J]

This is a very welcomed addition!

[Barugon]

What about animation blending (like Unity's Mecanim)?

[james7132]

What about animation blending (like Unity's Mecanim)?

You probably won't see a full animation controller like API anytime soon. However, the proposed animation graph API is low-level enough like Unity's Playables API, which supports arbitrary blending control, and you could potentially build a full state machine driven animation system on top of it.

[SamPruden]

Is the eventual intention to allow designers (i.e. people using the Bevy Editor, not coders) to apply arbitrary animations that drive arbitrary component values?

This is something that @james7132 brought up in the Schematics discussion https://github.com/bevyengine/bevy/issues/3877#issuecomment-1057752326. I'm wary of that, for reasons that overlap with the arguments made in favour of Schematics.

Firstly, there's the standard issue of tying animation assets too closely to ECS implementation details, meaning that refactorings to the ECS layout would break compatibility with authored animation (and other) assets. This is the problem that Schematics aim to solve, and may be addressable by some design in which animations drive Schematic values instead of component values directly. There's a little discussion of that in #3877.

There's also an ECS philosophy question that I'd be interested to hear the official stance on:

Should ECS Systems be expected to gracefully handle all typesafe Component values, even those that don't make semantic sense? Or alternatively, is it acceptable to rely on engineering knowledge about what values get set on the Component fields by the code? For example, should Systems processing a Component holding a direction vector be expected to have well defined and sensible behaviour if that vector is not in fact normalized?[^1]

On the one hand, in ECS it's hard to keep track of all of the places where a value is mutated, so one could argue that it's dangerous to make assumptions about data values. On the other hand, actually covering every corner case would have both engineering and runtime overhead. Inevitably, cases would be missed. In practice, I've never taken this rigorous approach.

The reason that this is relevant to animation is that if we allow arbitrary animations to make arbitrary changes to entity data, it becomes incredibly easy for these implicit invariants to be broken. The animations may be composed by people who are completely separate from the code, and who don't have any understanding of the restrictions and dangers. We don't want a scenario where somebody tries to animate a value and causes undefined behaviour because the value they applied was unexpected.

For these reasons, I think it's a bad idea for animations to directly touch ECS data without going through some validating mediation layer that only allows semantically valid animations. The ideas around Schematics may work for this, although I don't really know what that looks like yet.

What's Bevy's general philosophy to these problems around data validation and trust, and how that interacts with animations?

[^1]: This particular example might be a good candidate for newtype typesafety.

[james7132]

The intent is to have it work on Component that implements Reflect. For the most part, this usually implies that what you're writing to is mostly just read-only data that has no invariant to enforce (i.e. materials).