search

VoidXH / Cavern

Object-based audio engine and codec pack with Dolby Atmos rendering, room correction, HRTF, one-click Unity audio takeover, and much more.
http://cavern.sbence.hu
Other
307 stars 14 forks source link

Add ST 2098-2 Immersive Audio Bitstream #143

Closed izwb003 closed 4 months ago

izwb003 commented 9 months ago

Greetings! First I want to say thanks to your work, especially conversion from E-AC-3 JOC to ADM BWF in Cavernize. This is really a powerful tool for Object-Based Audio!

This issue comes from some ideas by myself. For many years we focus on Dolby Atmos for home playback and we have many powerful tools. But for Dolby Atmos theaters we are still unable to make Dolby Atmos contents that can be put into a DCP and playback in a Dolby Atmos Cinema. I believe that requires professional tools from Dolby.

However I am interested in how does a Dolby Atmos theater plays Atmos content, and recently I am suprise that I found out the new standard on immersive audio - Immersive Audio Bitstream from SMPTE, also known as SMPTE ST 2098-2, was adapted from Dolby Atmos Bitstream Specification, which was how Dolby Atmos DCP content was defined. As I have a copy of an unencrypted Dolby Atmos DCP, I extracted the Atmos asset and tried to analyze it - surprisingly found out it is almost the same as defined in ST 2098-2 (with some limits). That means the standard of DCP Atmos content is almost openly documented (I can provide you with more proofs and details if you are intersted in it).

So in this issue I would like to ask that whether you are interested in giving ST 2098-2 some research and make Cavern supports it? It might be helpful to the future as IAB is the future standard for cinema immersive audio.

Thank you and wish you a happy new year!

VoidXH commented 9 months ago

Dolby has a tool that converts back and forth between Dolby Atmos DCP tracks and ADM BWFs, called Dolby Atmos Conversion Tool. Because Cavern supports ADM BWF, including its non-standard Atmos version, for both reading and writing, any DCP is already supported for both encoding and decoding.

izwb003 commented 9 months ago

Dolby has a tool that converts back and forth between Dolby Atmos DCP tracks and ADM BWFs, called Dolby Atmos Conversion Tool. Because Cavern supports ADM BWF, including its non-standard Atmos version, for both reading and writing, any DCP is already supported for both encoding and decoding.

I use this tool frequently. However as I see, it can convert DCP tracks back into ADM BWF easily, but cannot convert anything into DCP tracks. If I import an ADM BWF, the options provided are ADM BWF, IMF IAB, .rpl and .atmos, without Cinema MXF.

Now I wonder any solutions about converting ADM BWF into Cinema MXF (DCP Content). It seems like we are still unable to make that happen. Do you have any ideas? Thanks.

VoidXH commented 9 months ago

It's IMF IAB.

izwb003 commented 9 months ago

It's IMF IAB.

Well, actually they are not the same thing...

IMF IAB is designed for a special kind of package called Interoperable Master Format. This format was adapted from DCP standard, but with some modification. IMF IAB was defined in SMPTE ST 2067-201 Immersive Audio Bitstream Level-0 Plug-in, while DCP IAB(Atmos) was defined in SMPTE ST 2098-2 Immersive Audio Bitstream Specification and with a limit to it - SMPTE RDD57 Immersive Audio Bitstream Application Profile 1.

So, this IMF IAB was adapted from DCP IAB (DCP Atmos), but they are not the same thing. DCP packaging software will not accept IMF IAB mxf and put it into a DCP. We still need to find a solution to make DCP IAB (DCP Atmos) tracks. Are you interested in this?

VoidXH commented 9 months ago

The doc looks straightforward enough.

izwb003 commented 9 months ago

The doc looks straightforward enough.

Thank you for your attention! Here I will explain what I've found. It may have mistakes but I hope they will be helpful.

Background story

Since Dolby Atmos was introduced by Dolby from 2012, Dolby began to sell their CP850 Cinema Processor, which can decode Dolby Atmos DCP tracks, which include the Dolby Atmos Bitstream. The specification for this bitstream is partially disclosed in SMPTE RDD29, but Dolby did not disclose the complete details.

Then the time came to 2018. During this period of time more and more immersive audio system appeared. They each have their own standards, which brings a lot of trouble for content creators to create content. SMPTE noticed this issue and 25CSS began to develop a unified immersive audio specification to encourage systems from different vendors to decode content produced according to this specification, to optimize compatibility. Then SMPTE 25CSS Immersive Audio Suite was published.

Due to the fact that Dolby Atmos is still the most widely used immersive audio system, the heart of 25CSS Immersive Audio Suite, SMPTE ST 2098-2 was adapted from the Dolby Atmos Bitstream Specification to make compatibility easier. It has added some features that may be used in the future. The bitstream defined by this document was called Immersive Audio Bitstream, aka IAB. All Dolby Atmos Bitstream in Dolby Atmos DCPs are IAB, and from then on Dolby Atmos DCPs' ISDCF DCNC name even changed from -Atmos into -IAB.

However, ST 2098-2 is very forward-looking as it contains many parts that cannot be processed by decoders already installed. So SMPTE published RDD57 IAB Application Profile 1 to limit ST 2098-2 as a constraint. In this file SMPTE mentioned: "Legacy content (Dolby Atmos) follows the guidelines of IAB Profile 1.", means that IAB compliant with RDD57 can produce Atmos DCP tracks that can be played in Atmos theaters.

For content creators IAB also helps. Creators may deliver IMF packages mentioned before. If IMF package contains Atmos, it contains IMF IAB, defined in SMPTE ST 2067-201, an extension to ST 2098-2 (please note that all IMF IAB created with Dolby's tools was modified by Dolby and does not strictly follow ST 2067-201. Dolby introduced their modification in this document).

Relationship between standards

ST 2098-2 defines IAB. IAB limited by RDD 57 is IAB Profile 1. IAB Profile 1 is the same as Dolby Atmos Bitstream for cinema Atmos DCPs. IAB extended by ST 2067-201 is IAB Level 0 Plug-in. IAB Level 0 Plug-in is IMF IAB. IMF IAB created with Dolby's tools are modified by the instruction from a Dolby's document.

Here is a video that provides a detailed introduction to the ins and outs of IAB.

If Cavern can handle it, what help will it bring

IAB seems like the future of immersive audio, at least I think so. As a Swiss knife-like tool to immersive audio, Cavern may be able to achieve encoding cinema immersive audio contents. For home entertainment users it may also be helpful, as IMF IAB is common to see. Providing support for IAB seems to have broad application prospects.

Sorry for writing a long paragraph, but it has already introduced all the information I know and I hope it can be helpful.

VoidXH commented 4 months ago

Priority is overridden by #151 and the first AC-4 content.