Upmix 2.0 to 5.1

[Obegg]

I've noticed that many of the video and audio files I play are 2.0 but I use surround sound system of 5.1. I was thinking if it was possible to upmix from 2.0 to 5.1 in mpv settings or user script? All I could find about it is this, but this is for Linux and not for WIndows.

Edit: I also want it to ONLY upmix when the audio file has 2 channels, no need to upmix when the audio source is 5.1.

Note: I have no issues playing DTS-HD MA or Dolby TrueHD files that have 6 or 8 channels, those are working fine, mpv and Windows recognize I have surround sound system.

[ghost]

[upmix]
profile-cond=p["audio-params/channel-count"] < 6
profile-restore=copy
audio-channels=5.1

As long as the layout is supported by your audio device, --audio-channels should work without having to mess with pan filters.

[Obegg]

This doesn't do any upmixing, by my understanding - --audio-channels only detemines which audio channels are accepted by mpv.

[ghost]

If setting the layout doesn't work then you'll have to apply an audio filter.

https://ffmpeg.org/ffmpeg-filters.html#surround

[upmix]
profile-cond=p["audio-params/channel-count"] < 6
profile-restore=copy
af=surround:chl_out=5.1

If you want to control the mixing of the audio channels individually, you'll have to apply a pan filter after upmixing the amount of channels from 2 -> 6.

[hooke007]

audio-params/channel-count is not reliable. https://github.com/mpv-player/mpv/issues/11541

[ghost]

The issue you linked seems to just be an issue with the order in which autoprofiles are processed (after everything in your mpv.conf has been set). Adding the downmix layout to the profile itself doesn't seem to cause any issues when switching between audio tracks mid-playback.

[louder_speaking]
profile-cond=p["audio-params/channel-count"]>2
profile-restore=copy
audio-channels=stereo
af-pre=@vocal:loudnorm

[louder_speaking-alt]
profile-cond=p["audio-params/channel-count"]<=2
profile-restore=copy
audio-channels=stereo
af-remove=@vocal

Also worth noting that this autoprofile quirk isn't really an issue with the default auto-safe anyways, or if you whitelist more than one layout.

[Obegg]

af-pre=@vocal:loudnorm

mpv gives me the error:

[csplayer] Option af-remove: item label @vocal not found.

Uhm?

The only thing that seems to be doing anything is:

[upmix]
profile-cond=p["audio-params/channel-count"] < 3
af=surround:chl_out=7.1
profile-restore=copy

[ghost]

Uhm?

If there is no audio filter prepended to the list then there's nothing to remove.

af=surround:chl_out=7.1

This would only make sense if the 5.1 audio is mapping to side channels, since a 7.1 upmix would output to side speakers. Does --af=surround:chl_out=5.1(side) do anything? Or better yet, --audio-channels=5.1(side)?

You could also play around with pan filters.*

af=lavfi=[pan=5.1| FL = FL | FR = FR | FC < 0.5*FL+0.5*FR | LFE < 0.5*FL+0.5*FR | BL=FL | BR=FR]

*not really an optimal example, this would probably be a loud upmix

Edit: Forgot to include front center channel to upmix, whoops

[Obegg]

I'll further explain my setup. I use Windows, I have 6 physical speakers (so 5.1), but on Windows I have the option to use 7.1 somehow so I chose it. Using audio-channels=5.1(side) seems to not play any stereo file, I have no idea why. Using af=surround:chl_out=7.1 seems to work fine and upmix successfully,

My only remaining questions are: 1) On Windows I can set up the audio as 7.1, but in actual speaker count I only have 6, so it's a 5.1 system, should I really choose 7.1 or should I choose 5.1? 2) The only thing that seems to be doing anything is af=surround:chl_out=7.1, I'm not familiar with audio filters and those commands, is this the most optimal and useful one or there are better ones?

[richardpl]

surround filter uses RDFT transforms and tries to extract soundfield from stereo audio, there are non-default options for this filter that give a better quality at cost of higher CPU usage. Other alternatives for upmix are simple one with pan filter, but that is just different linear mixes. Thats all that is currently available within mpv, unless you use LV2 plugins but that is mainly for Linux.

[Doofussy2]

You should set windows to 5.1 and the filter to correspond with the windows configuration.

[ghost]

On Windows I can set up the audio as 7.1, but in actual speaker count I only have 6, so it's a 5.1 system, should I really choose 7.1 or should I choose 5.1?

If you don't have a 7.1 system, then you shouldn't pick a 7.1 layout. You're physically missing two channels.

is this the most optimal and useful one or there are better ones?

It's more straight-forward than messing with pan filters at least. I would read the documentation I linked to see what options in the surround filter you can play around with, like smooth, angle, focus, win_func etc. Just a syntax example:

af=surround=chl_out=5.1:lfe=no:smooth=0.2:focus=0.2

It's something you'll have to experiment with since I don't have access to your setup. Leaving everything at default should work fine though.

[Obegg]

there are non-default options for this filter that give a better quality at cost of higher CPU usage

Throw it at me, I have a good CPU, let me use it to it max usage. Give me the commands to use.

You should set windows to 5.1 and the filter to correspond with the windows configuration.

Then why my AV Receiver says it supports 8 channels? Does this mean that if I choose 7.1 even if physically my system is 5.1 then the "missing channels" (Rear Right/Left) are duplicated to the surround speakers (Side Right/Left)?

If you don't have a 7.1 system, then you shouldn't pick a 7.1 layout. You're physically missing two channels.

Yes, but as I said above - they are getting duplicated, right? so I'm not "missing" them if my AV Receiver supports that, right?

[Doofussy2]

there are non-default options for this filter that give a better quality at cost of higher CPU usage

Throw it at me, I have a good CPU, let me use it to it max usage. Give me the commands to use.

You should set windows to 5.1 and the filter to correspond with the windows configuration.

Then why my AV Receiver says it supports 8 channels? Does this mean that if I choose 7.1 even if physically my system is 5.1 then the "missing channels" (Rear Right/Left) are duplicated to the surround speakers (Side Right/Left)?

If you don't have a 7.1 system, then you shouldn't pick a 7.1 layout. You're physically missing two channels.

Yes, but as I said above - they are getting duplicated, right? so I'm not "missing" them if my AV Receiver supports that, right?

You may be confusing all channel audio with surround sound. They are not the same. Surround sound does not duplicate audio fields, it analyzes the audio and tries its best to create discrete audio channels. Try listening to a music video with the surround filter, and you'll see what I mean. The main vocal will come from the center and the orchestral audio will be sent to the surrounding speakers. All channel audio is simply the same sound in every speaker.is that what you want? If yes, you should use the pan filter.

[richardpl]

Pan filter is only capable of producing linear combinations of input channels, so saying you should use the pan filter is invalid.

[Carl-Robert]

You should decide first if you want mpv to do the audio decoding or your AV Receiver.

If you want to use your Receiver's audio expansion/up-mixing features: Put Windows channels to your physical speaker configuration (5.1). Bitstream audio from mpv to your Receiver OR leave mpv to defaults if your Receiver doesn't support whatever audio format the files have (for example DTS-HD). In both cases your receiver should get the exact audio channels of the file with no changes. Then you can control the up-mixing or whatever on your Receiver without worrying about mpv settings. EDIT: Actually in this case bitstreaming works a lot better, because THIS is the case where you could configure Windows to the maximum channel count your Receiver accepts (7.1). My bad.

If you want mpv to do the audio up-mixing: Put Windows audio channels to the physical speaker configuration you have (5.1). Use the commented af=surround:chl_out=5.1 filter to up-mix to your physical channel count. Use some sort of "Direct" or other mode on the Receiver that does no channel up-mixing.

In any case Windows settings should match your physical speaker configuration. There is no upsides of forcing it to output 7.1 audio. When you want to skip Windows mixer the correct way is to bitstream the audio to the receiver OR decode the audio to the correct channel configuration on the software in question (there is no "double-processing" involved if mpv sends 5.1 to Windows mixer configured as 5.1).

[richardpl]

there are non-default options for this filter that give a better quality at cost of higher CPU usage

Throw it at me, I have a good CPU, let me use it to it max usage. Give me the commands to use.

Something like this with maxed focus:

surround=5.1:focus=1:overlap=0.875:win_size=16384:win_func=hann

Caveat: keep win_size power of 2 number (2048, 4096, 8192, 16384), and make overlap [1.0-(1.0/(2^n))] where n is small number (1-4) gives overlaps: (0.5, 0.75, 0.875, 0.9375)

bigger win_size means more detailed extractions of various cues. bigger win_size means you also need to increase overlap too otherwise you will get worse quality. By increasing overlap, needed CPU power for filter increases non-linearly.

Note that you need relatively recent ffmpeg version used by mpv for overlap=X option as previous version would give suboptimal results with non-default values.

[Obegg]

Thank you.

win_func=hann

Why this one and not others? (also - this is the default according to the docs, so why not just remove it from the command?)

So I decided to max out the parameters and use af=surround:chl_out=5.1(side):focus=1:overlap=1:win_size=65536, Played a stereo video file just for testing and CPU usage hasn't changed, still 5%, where is the "high CPU usage" you talked about?

need relatively recent ffmpeg

I'm always up-to-date - https://github.com/zhongfly/mpv-winbuild/releases

Edit: I'm noticing some audio delay, that's actually annoying

[richardpl]

That huge overlap and huge win_size are not very useful (unless you use very high sample rate, in which case you are not going to get much out of it), Just use parameters that I mentioned above. This is not toy product for average joe.

[Obegg]

Even with those parameters I have audio delay and I find it super annoying. Is there any way to counter it? The delay is a big issue for me. There's no delay without those parameters.

[Carl-Robert]

https://mpv.io/manual/master/#options-audio-delay Do use the manual. --audio-delay= with negative values combat it.

[richardpl]

How big is audio delay you are experiencing? MPV should automatically fix this...

[Obegg]

the audio delay is around 200ms, the reason I say "around" is because setting --audio-delay=-0.2 seems to fix it, but I have no tools to know the exact amount of delay, so this number could be off. mpv doesn't fix it automatically, which I find weird aswell, got any ideas?

Edit 1: Tested it even with the following command:

mpv "sync.mp4" --no-config --af=surround=5.1:focus=1:overlap=0.875:win_size=16384:win_func=hann

and there's still a significant audio delay.

[orion1vi]

This will cause other problems but you could try --audio-buffer=0

[richardpl]

Ah, the delay is introduced by filter itself, fixed it in my fork.

[Obegg]

Ah, the delay is introduced by filter itself, fixed it in my fork.

Can't find it. Also - will you send a PR to fix it in master?

Edit 2:

This will cause other problems but you could try --audio-buffer=0

@orion1vi --audio-buffer=0 did not fix the issue.

[ghost]

Also - will you send a PR to fix it in master?

For statically-linked builds, you will have to build mpv with librempeg to get the fork fixes.

[Obegg]

For statically-linked builds, you will have to build mpv with librempeg to get the fork fixes.

Sorry for the noob question: So I guess if I use shinchiro builds I am not getting the fix? Because according to this there's no librempeg. So... what should I do?

[ghost]

So I guess if I use shinchiro builds I am not getting the fix?

Correct.

So... what should I do?

You'll have to compile your own mpv build and impregnate it with librempeg instead of FFmpeg.

[Obegg]

Again - sorry for the noob question: What is the difference between them? As far as I know the FFmpeg is popular and many users and programs depend on it, I never heard of librempeg.

I want the audio delay issue to be fixed in FFmpeg, how can I do it in the most simple way? (Asking because I remember seeing the issue template and website and it looked so complicated)

[llyyr]

I never heard of librempeg.

Because it didn't exist until 48 hours ago. It's a ffmpeg fork by one of the currently former ffmpeg dev

how can I do it in the most simple way? (

Patch ffmpeg with https://github.com/librempeg/librempeg/commit/5d2a66861ef6c736545b80ca05732dcb72e84925

[Obegg]

Uh no, that's sounds like bad news. So just now I've connected to dots... (I did not look up who replied to me). Even if it's off-topic (and somehow still related to the original issue) I still have questions. FFmpeg is big and have many maintainers and users and programs and etc... librempeg supposed to be a fork, but... this means a few things: 1) Something made him leave FFmpeg, which I believe this is the relevant "patchwork", but I have no prior knowledge what happened there. 2) A fork means (I guess) to maintain it to a long duration, and not just the sections the he was "in-charge" of (if this is what it's called), that's a lot of work for a single person to do. 3) If it is supposed to be a "successor" to ffmpeg (which I guess it is supposed to be?), then it needs to gain fame, popularity, user base, programs that will depend on it, issue tracker, pull request tracker, maintainers, developers, it's huge amount of resources to be poured into it.

At the end of day, I'm a simple user, and I could be wrong with everything I just said (if I did - point it out and correct me), but I hope the situation gets well and peace will be restored (and I will get the fix I so badly need with the audio delay, because I have no idea to to build mpv other then downloading .exe file from github)

[ghost]

then it needs to gain fame, popularity, user base, programs that will depend on it

Soon.

[tramchamploo]

Anyone knows what the angle parameter is about? Why is it default to 90? I'm using hrtf to sofalize upmixed content. Some hrtf is measured at 270 degree. Is it related?

[richardpl]

Not really, angle option makes x/y polar coordinates more distorted (inward (< 90) or outward (>90)) so when angle > 90 it will make more content from stereo channels present in surround channels, back and side left/right and back center channels. Default 90 deg is normal mapping which will make differences normally propagated to all surround channels relatively equally. But default 90 deg is not gonna make big surround channels happy if mixing stereo image is very weak/boring (like almost mono mixing in stereo sound) so that is why this option exist.

[tramchamploo]

Thanks for reply. Now I know what the consequence this param has. Can you elaborate more on exact mean of this param and implementation detail? Does it dictate the angle BL and BR is from center channel? I tried to read source code but dsp is not something I'm good at.

[richardpl]

It does not dictate angle for BL/BR from center FC channel. Stereo audio have no angles for BL/BR channels, it just have two source channels. And this filter just maps stereo field that is created by FL and FR channel so that it copies/scales certain samples in frequency domain to other surround (SL/SR/BL/BR/BC...) channels. The angle and focus option are there to control how this transform from 2d stereo box is done to other channels, so kind of re-projection. Also please use smooth option value set to > 0 as it helps get rid of FFT related artifacts like noise tones or other unpleasant distortions.

You can not get from sofalizer/hrtf/binaural back to surround channels or reverse direction without some loss of data at all, this is always lossy process.