VoidXH / Cavern

Object-based audio engine and codec pack with Dolby Atmos rendering, room correction, HRTF, one-click Unity audio takeover, and much more.
http://cavern.sbence.hu
Other
320 stars 14 forks source link

E-AC-3 JOC file decode problem #47

Closed sclsj closed 1 year ago

sclsj commented 1 year ago

I tried to convert a few E-AC-3 JOC files (downloaded from Tidal) to ADM BWF wav, and I do not get the expected results.

Test track 1: Ado's Usseewa from Kyougen (jpop?) Expected: one channel contains drums, another contains vocals, another contains piano, another bass, etc, etc. Actual: every channel contains a different mix of all instruments (slightly different ratio). Some does only contain part of the instruments but it changes (i.e. for 30 seconds it's backing vocals + drums, then for 30 seconds it's backing vocals + synths, etc) The majority of channels are very quiet and when turned up sounds heavily distorted. Spectrogram shows holes and uneven lowpass.

Test track 2: Joe Hisaishi's Links from Minima Rhythm (orchestra, minimalism) Expected: one channel for violin 1, one channel for violin 2, one channel for cello, etc. Actual: every channel contains a different mix of all instruments (slightly different ratio). They all sound very similar. The majority of channels are very quiet and when turned up sounds heavily distorted. Spectrogram shows holes and uneven lowpass.

VoidXH commented 1 year ago

Please note that the home version of Atmos has a very small object number limit, and multiple objects have to be mixed into one. No content has separate instruments available in this format, only in the cinema. Distortions are documented in #17, but in most cases, the channels randomly swap around, the audio will follow on a different track. This is an encoder feature and helps lowering bitrate for the same quality, it's also observable for Dolby demos.

bluewidy commented 1 year ago

https://www.avid.com/resource-center/encoding-and-delivering-dolby-atmos-music

Since this describes the clustering that occurs during the ec3 encoding process, I expect it will address your curiosity about the phenomenon.

sclsj commented 1 year ago

https://www.avid.com/resource-center/encoding-and-delivering-dolby-atmos-music

Since this describes the clustering that occurs during the ec3 encoding process, I expect it will address your curiosity about the phenomenon.

Yes, seems like eac3 wouldn't be enough for my use. Are there ways you know of that I can get access to the source file (adm bwf)? My messages to the producers received no replies.

And since the article mentioned internet bandwidth as the reason for this compromise, since I have gigabit ethernet and it's spreading (or at least I hope so), do you expect to see the source file available for consumer streaming/download happening any time near the future?

sclsj commented 1 year ago

Test track 2: Joe Hisaishi's Links from Minima Rhythm (orchestra, minimalism)

That is fake atmos, no objects actually move.

Test track 1: Ado's Usseewa from Kyougen (jpop?)

That is real Atmos.

OK, that makes more sense now. Well, they are not really different; they are both not good enough for my use. By the way, are there ways you know of that I can get access to the source file (adm bwf)? My messages to the producers received no replies.

VoidXH commented 1 year ago

Cavern driver has some speed (realtime) problems

That's an IO bottleneck, it can handle 160 objects running from memory on a Ryzen 5600, that's more than 1 full track per core. For me, playing the content from an SSD fixes all perf issues.

do you expect to see the source file available for consumer streaming/download happening any time near the future?

Never, because it's useless for home use. We don't have 64-channel systems to take advantage of it, a 9.1.6 downmix is actually way more than enough in most cases. The concern is local storage space, not network space. An 18 Mbit/s downmix sounds exactly like the 250 Mbit/s raw for low channel counts.

bluewidy commented 1 year ago

Are there ways you know of that I can get access to the source file (adm bwf)? My messages to the producers received no replies.

It's simple. Just contact the immersive audio mix engineer responsible for the project. But this is an act of asking for a project file. In other words, you are telling the chef to give me a recipe. It may be a slightly difficult request, but if you explain well why you want the adm bwf file, they can provide you.

Ado's Usseewa from Kyougen

Contact Kenichi Koga

Joe Hisaishi's Links from Minima Rhythm

Contact Suminobu Hamada

sclsj commented 1 year ago

Are there ways you know of that I can get access to the source file (adm bwf)? My messages to the producers received no replies.

It's simple. Just contact the immersive audio mix engineer responsible for the project. But this is an act of asking for a project file. In other words, you are telling the chef to give me a recipe. It may be a slightly difficult request, but if you explain well why you want the adm bwf file, they can provide you.

Ado's Usseewa from Kyougen

Contact Kenichi Koga

Joe Hisaishi's Links from Minima Rhythm

Contact Suminobu Hamada

@bluewidy

I couldn't find their contact information (e.g. email) anywhere. I contacted the respective label / collaborating company again a long time ago and received no replies. What can I do next?

The reason I want it is similar to how it's explained in MPEG-H SOAC white paper, so perhaps this might be a justified request. What do you think?

Interactive Mixing: Consumers can create personal interactive remixes using a virtual mixing desk. For example, certain instruments can be attenuated for playing along (similar to Karaoke), the original mix can be modified to suit personal taste, etc.

One specific issue I'm having is that it's too much bass and kick for me (common issue in pop), and EQ doesn't seem to help very much.

bluewidy commented 1 year ago

If so, I think you should give up. Labels and collaborators are not charities, so they have no reason to help you. In addition, project files are an asset containing their know-how, so they cannot be shared carelessly.

But if you're a remixer, try explaining who you are to them. If you're lucky, they may be interested in collaborating with you.

VoidXH commented 1 year ago

One specific issue I'm having is that it's too much bass and kick for me

Spleeter can help you to separate it so you can remix it.

sclsj commented 1 year ago

If so, I think you should give up. Labels and collaborators are not charities, so they have no reason to help you. In addition, project files are an asset containing their know-how, so they cannot be shared carelessly.

But if you're a remixer, try explaining who you are to them. If you're lucky, they may be interested in collaborating with you.

That is one major concern I have before I sent out the requests. Although I did purchase their albums / tracks online, and are willing to give them more money in exchange for the tracks in production format, it is probably too small of an economic incentive for them to care especially considering how much sale they are making.

Another important reason is that most releases today are heavily DRC / peak limited and no alternative versions are available for purchase. If I have the production format I can work around that issue. Maybe this would be an easier thing to ask for, although I also got no replies when I ask about it [ask for purchasing a version with no DRC or a diff file]. Not sure what can I do here either.

Unfortunately I'm just an average consumer (with a bit more special needs) so collaboration is not an option.

sclsj commented 7 months ago

If so, I think you should give up. Labels and collaborators are not charities, so they have no reason to help you. In addition, project files are an asset containing their know-how, so they cannot be shared carelessly.

But if you're a remixer, try explaining who you are to them. If you're lucky, they may be interested in collaborating with you.

@bluewidy Just wanted to ask some follow up questions. In what circumstances might I present enough incentive for them to give me the file? What it be the case that a single consumer will never be able to offer enough incentive and I have to find another big company as a proxy for me if I want some rate of success?

bluewidy commented 7 months ago

I don't know either. I think... using artificial intelligence to extract multiple stems from a stereo track is a realistic option you can implement. https://www.lalal.ai/