Standards compliant 3-D audio for Csound

[gogins]

It's now possible to get HRTF-decoded audio from Dolby Atmos or Sony 360 Reality Audio from Amazon Music on smartphones. I finally turned on Atmos for my Amazon Music Android app and it sounds great. It really does work. Of course this stuff has been in movie theaters for years. Unfortunately neither Linux, nor open source software, receive any attention from Dolby or Sony.

This issue can be resolved either with new plugin opcodes or by the use of existing LV2 or VST plugins.

On Apple one can buy VST3 plugins etc. for 3-D audio, and that would work with Csound.

The underlying standard for Sony is MPEG-H 3D Audio. The standards can be accessed at https://mpeg.chiariglione.org/standards/mpeg-h/3d-audio and https://www.atsc.org/atsc-documents/a342-part-32017-mpeg-h-system/. Dolby Atmos does not use MPEG-H. Note, the Motion Picture Experts Group was disbanded last year.

My impression of Csound is that the developers have failed to recruit new talent and are either in maintenance mode, or else focussed on Csound 7 which is a purely internal set of improvements. After reading the very good Sound on Sound article on immersive audio and glancing at the MPEG-H 3-D Audio standard, I believe implementing this using the current Csound developers is not feasible.

I am searching for any Linux software at all that implements MPEG-H, 360 Reality Audio, or Dolby Atmos for panning and rendering. I don't find any. But there is plenty of Ambisonic, VBAP, etc. What I need there is either easy to use libraries, or VST plugins that work on Linux.

Here is a good list of resources: https://wesoundeffects.com/ambisonics-and-spatial-audio-freeware-for-your-projects/.

• [x] IEM Suite. Ambisonics only, VST.
• [x] Sparta, VST.
• [ ] ambiX, another Ambisonics suite, VST, not currently building on my Linux computer.
• [x] MPEG-H reference software? Not on the MPEG site.
• [x] Blue Ripple, VST2 but only on Windows and macOS.
• [x] Ambisonic Toolkit for Reaper and SuperCollider, no VST.
• [x] IRCAM? A bunch of stuff for Ableton or Max/MSP, and this for Reaper, only for Windows and Mac OS.
• [x] SAF, an Ambisonics toolkit, used for the Sparta plugins.
• [x] spatialaudio.net, server that uses Jack.
• [x] VISR Production Suite uses objects as Atmos and 360 Reality do, ISC open source license, looks like Windows and macOS only.
• [x] envelop for Live.
• [x] Facebook 360, looks like Windows and macOS only.
• [x] SpatGRIS3, looks like Windows and macOS only.
• [x] spatium, standalone or Max only.

[gogins]

Conclusion: I am falling behind the curve, as is Csound. The best things for me to do:

• [ ] Get a Macbook.
• [ ] Port my software.
• [ ] Ensure that VST plugin support in Csound is solid and cross-platform. At some point, there will be immersive audio VST plugins that will be affordable and that can be used in Csound. This is better than re-inventing the wheel inside Csound. The Clang opcodes if accepted could also be a good choice.

[gogins]

• [x] Install IEM suite and SPARTA plugins.
• [x] Instantiate in Csound and print out plugin information.
• [x] Determine signal flow graph for IEM suite and SPARTA plugins. Can't figure out VST parameter mappings for SPARTA's COMPASS plugins, which are the best sounding, as source code does not seem to be available.
• [x] Establish how VST routes spatial audio in Csound. Audio and Ambisonic audio channels are multiplexed in 64 channel input and output audio buffers in the VST process methods.

[gogins]

There is a problem in vst4cs opcodes with audio input and output types for the vstaudio opcode. It was designed with only up to 32 discrete channels in mind. What is needed is up to 64 channels of both input and output in the form of arrays ("a[]", not "y" and "mmm...").

The signal flow graph opcodes have array forms, but the loops are unrolled over the entire array. The shape of an array is determined by introspection.

• [x] I will need to create vstaudioa (but not vst3audioa for the vst3-opcodes, as VST3 supports fewer channels).

[gogins]

Done. Plugins should generally be preferred to builtins for spatialization as the intellectual capital on the plug-in side seems much more committed than that on the Csound side.