VoidXH / Cavern

Object-based audio engine and codec pack with Dolby Atmos rendering, room correction, HRTF, one-click Unity audio takeover, and much more.
http://cavern.sbence.hu
Other
321 stars 14 forks source link

Questions about Limitless Audio Format #186

Closed ThreeDeeJay closed 3 months ago

ThreeDeeJay commented 3 months ago

I brought up the possibility of using the LAF format to record object-based audio (like, from games) using OpenAL Soft and the dev seems interested in it, though he has technical questions I'm not sure I can answer:

That is extremely interesting. Particularly the point about "A PCM block is created for each second of audio. [...] For object-based exports, extra PCM tracks are used for object movement data." Does that mean the dynamic object positions only update once a second? That doesn't seem correct, given the atmos demo (and that video's conversion/rendering of it) seems to update more frequently/smoothly. Unless the idea is the sound moves over the course of that second, which would make it more practical to deal with, though pose problems for outputting it in real-time. Quick fine movements would be lost, and there may be a delay in the position updates.

VoidXH commented 3 months ago

The updated positions are written to extra PCM tracks, which means, the update frequency is dependent on the sample rate. One such track holds positions for 16 objects (for the next 16 or less, another track is created, and so on), and one update takes 3 samples (X, Y, Z coords with the precision of the selected bit depth). This results in an object update every 48 samples, so 1000 updates every second at 48 kHz, which is 32 times faster than Atmos.

VoidXH commented 3 months ago

The updates happening every second are related to track availability blocks. They disable PCM export for tracks containing only silence for the next second. Objects playing audio are re-enabled. This saves a lot of space when dealing with hundreds or thousands of objects that rarely play audio.

For a reference implementation of exporting already created playback environments, see LimitlessAudioFormatEnvironmentWriter.cs.

ThreeDeeJay commented 3 months ago

Thanks, mate! I'll pass the info along.👌

ThreeDeeJay commented 3 months ago

By the way, the dev has some feedback that seems like a good idea so I figured you might want to know/respond

Also interestingly, this LAF doesn't seem too well optimized. Each chunk always has all tracks enabled, when I'm sure many of them are silent for long stretches of time. I also wonder if there's room to optimize track utilization (if two or more tracks don't play sound at the same time, they can be on the same track; you'd only need as many tracks as are played simultaneously). Those would help cut down on the LAF file size pretty well, I think.

VoidXH commented 3 months ago

Thanks, I'll answer it.