search

bevyengine / bevy

A refreshingly simple data-driven game engine built in Rust
https://bevyengine.org
Apache License 2.0
36.1k stars 3.56k forks source link

Add support for seeking audio. #9076

Open Gnamra opened 1 year ago

Gnamra commented 1 year ago

What problem does this solve or what need does it fill?

Seeking audio is practically a must have for any rhythm game that wants to have a beatmap editor. In the song selection screen of a rhythm game, it is also common to play a preview of the song you have selected. Usually this preview starts somewhere in the song, and not from the beginning.

What solution would you like?

If I'm not mistaken bevy_audio uses Symphonia and Rodio. Symphonia already supports seeking, exposing this would be enough for my use case. Additionally, it would be nice to be able to get total playtime of the audio file since that would make it easier to make decisions about where to start previewing a file.

What alternative(s) have you considered?

Setting the playback speed very high to advance to the point you want to seek to in a single frame, but that's not very elegant. bevy_kira_audio is the only reasonable alternative right now as far as I know.

harudagondi commented 1 year ago

Bevy uses Symphonia indirectly, and Rodio has not exposed API for seeking. Please see:

Currently Bevy is blocked on this.

shimekukuri commented 9 months ago

(Update) Rodio feature Audio Seek awaiting review and pull request.

See: https://github.com/RustAudio/rodio/pull/513

dvdsk commented 7 months ago

has been merged :partying_face:

l will cut a release in a few days but you should be able to start building a PR on top of it now.

Gnamra commented 6 months ago

@harudagondi Since we are no longer waiting for rodio, can the blocked label be removed?

SolarLiner commented 6 months ago

I've removed the blocked label, so anybody feel free to tackle this!

It should be noted that work on this will soon be superceeded by the ongoing work to replace rodio with kira, so it has a limited shelf life; however contributions are still welcome.

dvdsk commented 6 months ago

It should be noted that work on this will soon be superceeded by the ongoing work to replace rodio with kira, so it has a limited shelf life; however contributions are still welcome.

Out of interest, (I now help maintain rodio) could you point me at the reasons why bevy is switching to kira (kira looks great btw)? That could help plot a course for rodio's development.

SolarLiner commented 6 months ago

Out of interest, (I now help maintain rodio) could you point me at the reasons why bevy is switching to kira (kira looks great btw)? That could help plot a course for rodio's development.

rodio has some fundamental downsides that makes it unfit for use in Bevy as an audio engine:

  1. The main abstractions are built around the Iterator trait, which forces some suboptimal design decisions (ie. no easy way to broadcast signals to multiple outputs for creating send effects, or lack of metadata about the iterators being infinite or not).
  2. Having to wrap a Source with an effect to apply the effect is cumbersome and, since it involves the type system, cannot be done dynamically unless all combinations are present at all times. This hinders attempts at building dynamical audio systems, which might want to add and remove processing at runtime (ie. reverb volumes in games, or only processing a low-pass filter under water), or mixing multiple sources into a single track to do bus processing instead of wasting resources processing N effects in parallel for each source.
  3. The critical audio path is not real-time safe, which is a big downside for an audio library, as audio processing is at the mercy of the OS descheduling the audio thread, resulting in crackles from underruns, and choppy audio playback.

Kira specifically has some nice-to-have features that make it more attractive to base Bevy's audio engine around:

  1. Kira has a mixer feature, with flexible routing, which is much more condusive to usage in games. Its tracks are the place to put effects, and tracks can be created at runtime, which means you can both do bus processing and individual effects processing as well. It can also add effects on the master track, which is an important last step (ie. using compressors and limiters to automatically adjust the volume level and prevent clipping of audio samples)
  2. Effects and tracks can be controlled separetely from the audio thread structures, which means not having to deal with explicit synchronization; instead Kira uses controllers with command queues that are applied on each audio callback, and supports sample-accurate automation by providing an instant at which to apply the change, or locking to a clock for musical time-based events, which is going to be useful for rhythm games. This also means that it's easier to integrate into the ECS world by attaching the sources, effects and tracks controllers as components.

One downside of Kira at the moment is the lack of runtime changes to the track effects rack or the output of audio sources into the mixer, which is a bit of a bummer, but I'm more confident with the architecture of Kira that it can eventually be added.

Sorry it's bleak, and as a maintainer it's not the easiest to read, but that's my feelings and the reasoning behind the change.

dvdsk commented 6 months ago

@SolarLiner Thanks for you for taking the time to write that very clear explanation. I had a look around the Kira source and it looks great. I'm gonna contact its maintainer and discuss kira's scope. If everything rodio provides and my future plans for it fit into kira better then it could be better to move my focus there. It makes little senses to split the rather small rust audio contributions between two similar crates.