MVP for Audio Playback

[jleibs]

Context

There's a much more detailed request for a broader set of audio features here: https://github.com/rerun-io/rerun/issues/2852

This issue is about a more minimal feature to just get audio playback working in the most minimal capacity.

Proposal

Make it possible to log audio assets as a single (or small number of) blobs and then have these play back only during "play" mode in the viewer. Avoid the hard questions around hings like audio-packetization, time-spans, and cross-timeline synchronization.

Simplifying assumptions:

• The timestamp of the audio asset is the time at which audio should start playing on the given timeline.
• The audio file should only be logged to timelines where the time scale matches the native audio time scale of the file.
• Only support playing back one audio stream at a time.
• Only play back audio when specifically in "play" mode with a 1.0 time-scale.

Suggested implementation

• Add the ability to log a binary blob containing audio data in a small number of common formats using a similar approach to Asset3D.
• Create an Audio "space view" which only supports a single audio asset. The view probably just shows something like an audio icon and maybe a volume / mute control.
• When scrubbing time, the view will load the approach "LatestAt" audio payload given the current time selection, and then offset the stream to the correct relative timepoint.
• When time is "playing" the view will playback the audio file using the native clock and track to make sure the audio clock and the native clock continue to be aligned.

Crates

• https://github.com/mrDIMAS/tinyaudio
• https://github.com/RustAudio/cpal

[vijay609]

Thank you @jleibs for discussing this and taking time out to create the issue. This level of audio support is going to help us get unblocked for our project and really centralize ReRun as "the" visualizer.

[emilk]

I created a small proof-of-concept branch experimenting with this a while back. It is not mergable, but may be inspirational for when we decide to work on this for real:

https://github.com/rerun-io/rerun/compare/main...emilk/audio-poc