Similarly to how audio sources are currently integrated, tracks can be added in two steps:
A first struct which details the track setup is added to the ECS by the user through a TrackBundle.
A ECS system reacts to additions of this component to create the Kira track with the specified properties.
Due to a limitation with Kira, tracks cannot be updated after they're created, so the original struct cannot be used to update the track live (needs Kira issue to change this)
Track handles should be wrapped in a newtype component to let users change the track after it has been created.
Effects
Tracks in Kira have an effects rack, a list of effects that process the sound coming into the track sequentially. Support for effects is a highly requested feature, so the API for it should be as intuitive as possible.
Effects should ideally be thought of as entities as well, which simplifies access from systems. They need to be attached to a track, which eventually can become a relationship once those land in future versions of Bevy; however for now, the only feature we can reliably use is the parenting relationship system; so for now, effects can be implemented as being children entities of the track entity they're on.
Integration of effects into the ECS is done is much of the same way as audio sources. Since there are multiple types of effects, each strongly typed, there should be a generic plugin that takes care of the integration.
The user would specify a settings component through an EffectBundle, which would then apply the effect to the parented track. An EffectControl component is then made available once the effect is added to the track, making it available to be controlled by other systems.
Another downside of the approach is that effect ordering is not explicitly given; insertion order can be recovered from the entity ID, which should be monotonically increasing in terms of its Ord implementation. This is fine, as there are no ways to either reorder effects or any way to insert any post-creation (see Limitations below).
Limitations
Tracks and effects in Kira cannot be "structurally changed" after being created, ie. you cannot add or remove tracks, change their routing, or add or reorder effects after the track has been created. This means that all track-related setup needs to happened within the same tick, which is unfortunate and the path forward most likely warrants asking for more runtime control from Kira.
Possible alternatives
"Render graph"-like setup
Another possibility is to provide a plugin build-time API for creating tracks and effects. This would disallow adding new tracks after the fact, but lends itself to Kira's way of working better. After build, the mixer structure is frozen, and one can only interact with tracks from a Mixer resource which owns all track and effect handles. This also means that audio systems modifying tracks and effects essentially cannot run in parallel, but also simplifies the implementation.
Effects as part of the track
The track can "own" the effects, which simplifies integration and setup, but means all queries changing effects need to have exclusive access over audio track components, which limits the parallelization the scheduler can do over audio-centric systems. It allows a more explicit ordering of effects though.
Summary
Integrate tracks as entities in Bevy.
Implementation proposal
Tracks
Similarly to how audio sources are currently integrated, tracks can be added in two steps:
TrackBundle.Track handles should be wrapped in a newtype component to let users change the track after it has been created.
Effects
Tracks in Kira have an effects rack, a list of effects that process the sound coming into the track sequentially. Support for effects is a highly requested feature, so the API for it should be as intuitive as possible.
Effects should ideally be thought of as entities as well, which simplifies access from systems. They need to be attached to a track, which eventually can become a relationship once those land in future versions of Bevy; however for now, the only feature we can reliably use is the parenting relationship system; so for now, effects can be implemented as being children entities of the track entity they're on.
Integration of effects into the ECS is done is much of the same way as audio sources. Since there are multiple types of effects, each strongly typed, there should be a generic plugin that takes care of the integration.
The user would specify a settings component through an
EffectBundle, which would then apply the effect to the parented track. AnEffectControlcomponent is then made available once the effect is added to the track, making it available to be controlled by other systems.Another downside of the approach is that effect ordering is not explicitly given; insertion order can be recovered from the entity ID, which should be monotonically increasing in terms of its
Ordimplementation. This is fine, as there are no ways to either reorder effects or any way to insert any post-creation (see Limitations below).Limitations
Tracks and effects in Kira cannot be "structurally changed" after being created, ie. you cannot add or remove tracks, change their routing, or add or reorder effects after the track has been created. This means that all track-related setup needs to happened within the same tick, which is unfortunate and the path forward most likely warrants asking for more runtime control from Kira.
Possible alternatives
"Render graph"-like setup
Another possibility is to provide a plugin build-time API for creating tracks and effects. This would disallow adding new tracks after the fact, but lends itself to Kira's way of working better. After build, the mixer structure is frozen, and one can only interact with tracks from a
Mixerresource which owns all track and effect handles. This also means that audio systems modifying tracks and effects essentially cannot run in parallel, but also simplifies the implementation.Effects as part of the track
The track can "own" the effects, which simplifies integration and setup, but means all queries changing effects need to have exclusive access over audio track components, which limits the parallelization the scheduler can do over audio-centric systems. It allows a more explicit ordering of effects though.