toby-yoyo
commented
2 months ago Various things have changed due to this task:
audio_channel_num
- This function will respect the number passed into it.
- An internal lower limit of voices has now been removed, so users can now pass 0 to the function to disable audio playback.
- Channels used to play sync groups are now part of the pool that is resized by this function.
- Note that audio from video playback will still not be affected by this function on most platforms.
audio_debug
- Two new source states will be displayed in the audio debug window:
- 'Waiting' sources are preparing to play and will begin playback once some internal conditions have been satisfied. These are coloured in magenta.
- 'Fading' sources will stop after the next mix occurs on the audio thread. These are coloured in cyan.
- Note that both of these states are transient and will probably only be visible for some time in the order of tens of milliseconds.
- Individual sounds played in sync groups will be visible in this window (see below).
- The
numProcessedcolumn has been removed as it will only ever display0due to changes to when audio buffers are (un)queued. - A
syncSourcecolumn has been added which shows which source each source is synchronised to. This will display-1for sources that are not syncing and for sources that are being synced to by others.
Audio sync groups
- Sounds in sync groups will now each have their own source. Previously, they were premixed and routed to a single source.
- Sources assigned to sounds in sync groups will no longer be separate from the main voice pool, and so they will be affected by the same rules and behaviours that regular sounds are (voice stealing,
audio_stop_alletc.). Note that sync group sounds are currently played with the maximum possible priority. - Sounds in sync groups no longer need to be exclusively compressed - a mixture of compressed and uncompressed sounds can be played together in a sync group.
- Sync groups will now be supported on the GX.games target.
- Sync group functions taking a sync group index can now print 'bad index' warnings.
audio_sync_group_debughas been stubbed. Useaudio_debugto debug sounds played together in sync groups.
gurpreetsinghmatharoo
alicemoretti
scott-dunbar
Is your feature request related to a problem?
The primary aims of the audio decoding threads are:
Our decoding threads do achieve these things. However, they also do some other things, such as:
The result of this is that many simple processes in the audio engine (e.g. stopping a sound) become an async process where we have to defer actions to the decoding threads. On top of this, we need to present these async processes as synchronous to users, so we have to build up rather complex systems to achieve this.
Additionally, the decoding threads have a few problematic behaviours such as 'detecting' buffer underruns, leading to potentially incorrect source state overrides, and partial deinitialisation of voice subsystems, which steps on the feet of the main thread. There is also a tight coupling of the stream states and source states which doesn't make much sense as these are two mostly unrelated systems.
Finally, we have at minimum 6 different threads that access the audio context (plus more depending on other features that users may/may not be utilising). This presents problems such as increased contention for the context (which is handled by only a single mutex), and loss of error state (which is intended to be read from a single thread).
Describe the solution you'd like
The decoding threads already do the main things they are supposed to, and generally in a reasonable way as well. However, moving the responsibility of certain tasks to the main/audio threads will help to untangle the code somewhat.
We should decouple the decoding threads entirely from the audio context. Decoding threads should not really need to know about or interact with the audio context, or even the engine more generally as they are essentially a specialised file/stream reader.
Specific changes we would make here would include:
Additional context
This isn't a huge change, but it should help us towards cleaning up a more awkward aspect of our voice handling. Improvements to voice handling more generally will help us to pave the way for features that rely heavily on voice management, such as audio scheduling. This in turn will allow us to address issues in existing features such as audio queues and sync groups, which can only be used in limited ways and are mostly isolated from the other parts of the audio engine.