Storyboards Epic

[xen2]

Have an animation editor to easily create cut-scene or animate properties.

TODO: Had some specs in the past, need to find them for future reference.

[DW01]

What sort of system would we want to aim for with this? UE4 Sequencer is still in early access but I've heard a lot of good about it on their forums, and it essentially functions like a normal NLE (or, to use a better analogy, kind of like an improved, in-engine Source Filmmaker). Do we want track-based editing (of properties, animation components, etc..) or something else?

Needless to say, we could also easily leverage ffmpeg (or even just the newly-implemented Video component) to allow exporting of Storyboards out to image sequences, uncompressed video, etc..

[xen2]

Ok, I am going to post here some confluence doc we made at the time. Not saying this is exactly what we'll do, but it should give a good overview of what we had planned some time ago:

Motivation

• Authoring and composition of time-based sequences (cutscenes, replayable bahaviors, etc.) that drive arbitrary properties of objects
• Manipulation of involved animations (changing speed, cutting, possibly crossfading)
• Editing of time-based properties as 3D curves the editor (mostly transformations, possibly more)
• Live preview of storyboardsDesign notes

Use-cases

• Cutscene where one story track will govern many aspects of the game like animations, curves, properties, etc.
  • Example: Cutscene where multiple characters interact with the environment and each other, possibly spawning visual and other effects
• Looping or non-looping track where some properties are animated or events are spawned
  • Example: A slowly spinning waterwheel produces splash sounds and effects
• Event list sampled based on animation tracks, so events can be spawned corresponding to the animation state
  • Example: When an animated character runs, climbs or punches, sounds and visual effects are spawned
• Animating one UIElement with different properties
  • Example: Ease-in ease-out type of animation is played on a button every time the button is pressed

Design notes

Model

• Storyboard
  • A Storyboard is authored as a StoryboardAsset and describes some time-based progression
  • The asset does not depend on a scene. It describes some time-based progression. Objects involved are not referenced by the storyboard, but bound when instantiating it
• Tracks
  • Each storyboard consists of a collection of tracks, each describing a single property or logical sequence
  • The target of a track is identified by a property path. The target may contain
    • "slots" for a top-level objects of a certain type, which can later be bound to actual object instances
    • a nested members, component (for objects), etc.
    • custom identifiers (not describing a property of an object), that can be used by other tracks for composition, or in script to query custom values
  • A track can contain anything that implements some understood interfaces. This should support
    • Curves that can be randomly evaluated
    • Skeletal animations (later possibly whole blend trees)
    • Sounds (playing, stopping, seeking?)
    • Events (fire-and-forget and start-stop)
    • Particles (visual script-based?)
    • other Storyboards
  • Tracks might have push or pull semantics depending on the target
• Binding of objects can be done by code, in visual scripts or creating a StoryboardComponent (and setting a Storyboard and setting up bindings) at design time

Run-time

• StoryboardComponent
  • References a storyboard and manages object binding and playback state
• AnimationComponent
  • might change in the future (possibly being responsible for animation blend trees?)

Design-time

• Timeline editor
  • Tracks contain blocks/markers of compatible type
  • Tracks might be displayed as tree view, when target path have parent-child relationship
  • Curves might be special in the way they they are previewed (directly in the track). They can be edited in the curve editor.
  • Animatable properties are automatically discovered and suggested (similarily to visual scripting, etc.)
• Storyboard preview
  • The storyboard can be previewed and played in custom 3D editor. A preview scene can be seleted and bound to the storyboard. It can then be edited using the bound objects from that scene.
  • Transformation curves are represented by 3D splines and control point gizmos. All common control-point types are supported (both here and in the 2D-curve editor).
  • The preview has video-like playback controls (play/seek/reset)

Further investigation

• Non-linear behaviors in the timeline
• Interaction with graph-based behaviors, e.g. blend trees (composition in both ways Storyboard <--> graph)
• UI storyboarding

Time estimate

• Timeline editor
  • 1w Track Control: Creating the controls to display and interact with a track (placing/moving keys, presenting key content. etc.)
  • 1w Track add/remove editor: Container of the tracks, support add/remove, possibly reorder, etc.
  • 1w Curve editor improvements: Improving performances and UX of the curve editor
  • 2w Track interfaces: track interfaces and their view model counterpart
  • 2w Property grid: drag & drop, display property of a track group, etc.
  • 1w Timeline: Control the vertical time cursor, seek, play, etc.
• Preview
  • 3w In-scene editing: Implementing an EditorGame for the scene
  • 2w 3D tracks: Display splines in 3d, allow to modify them
  • 1w Play/seek/reset: model-side playback logic
  • 1w Camera behavior: Custom tools to edit a camera track
• Runtime/model
  • 1w StoryboardAsset (and models for curves/events/scripts)
  • 3w UpdateEngine integration (pushing/pulling updates)
  • 1w Property discovery

[everyoneishappy]

Just wanted to check if the status of timeline tooling is is still as above?

[meriaizen86]

Just wanted to check if the status of timeline tooling is is still as above?

There is something for this fork.

https://github.com/phr00t/FocusEngine/blob/master/sources/engine/Xenko.Engine/Cinematics/CinematicAnimation.cs

Thank you for your interest.