Audio channel downmix 6->2 works, 6->4 just discards channels

[rvalles]

When playing a 6ch audio track with -channels 4, instead of mixing down to 4 channels, it uses 4 and discards the other two. This results in very awkward sound (like missing voices, which would come from the front centre channel in a 6ch setup).

Note this doesn't happen with -channels 2, where a decent stereo mix is obtained as expected.

[ghost]

The -channels option requests decoder downmix, so it depends entirely on the audio decoder (libavcodec) what happens. -channels 2 is a special case, and the only case that triggers actual mpv-specific downmix. Currently -af pan can be used to remix channels, but it's very "manual" and comes with a complicated command line syntax.

Note that -channels 4 will request 4.0 audio from libavcodec. From what I know, 5.1->3.1 would make much more sense (and 5.1 is the most common layout for 6 channels). But currently no way to request a specific channel layout through mpv.

With what audio codec does this happen? What is your OS and AO? Could you explain your use case? (I don't know too much about surround audio setups.)

By the way, there is a branch that adds support for channel layouts and automatic up- or downmixing to mpv, and I hope it will be merged soon. This branch doesn't change the -channels behavior yet, so suggestions how this stuff should work are welcome.

[rvalles]

That's interesting (about there being a branch actively working on this)

What's expected from an user pov is to be able to say "I have a 4.0 speaker setup" (or whatever applies to the user) and then mpv figuring out what's best at each time. A reasonable expectation for this example includes speaker fill (that is, mono source to all channels, stereo source front copied to rear), identity if the source is 4ch, and proper pan to 4ch if the source is 6ch or 8ch.

My use case is trying to play a movie that has a 6ch FLAC audio track on a 4 speaker setup, with -ao alsa. What's expected is some reasonable automatic panning to 4 channels. What happens is audio is a mess, most prominent issue being that I can't hear voices or they're very quiet (as they're in the front channel, which seems to be discarded).

As far as I can tell, it also happens with DTS and other codecs as long as there's a >4ch layout.

[ghost]

That's interesting (about there being a branch actively working on this) What's expected from an user pov is to be able to say "I have a 4.0 speaker setup" (or whatever applies to the user) and then mpv figuring out what's best at each time.

I wonder if the -channels option should always remix audio to the given number of channels, or more exactly: the channel layout, e.g 4.0. But I guess you don't actually want stereo to be upmixed to 4.0. So there are certain cases when no remixing is preferable. How should this be decided? (Here's a graph that shows the unexpectedly complicated relations between channel layouts; an edge A->B means all speakers in B are contained in A: http://ompldr.org/vaTE1dw)

The branch I mentioned is work in progress. You can try the branch named audio_stuff5 and run mpv with the following command line: mpv sometestfile.mkv -af force=channels=4.0 -channels 8. The -channels 8 is only for disabling stereo-downmix. libavresample/libswresample is used to do the remixing. I'm not sure if these libraries do it correctly, or in a way the user would expect.

6ch FLAC audio

FLAC doesn't support decoder downmixing, so using -channels 4 disables stereo downmixing, and has no other effects. mpv (and MPlayer etc.) will try to pass through the audio unchanged, so ALSA is actually opened in 5.1 mode. I guess ALSA doesn't try to remix the audio on its own.

As far as I can tell, it also happens with DTS and other codecs as long as there's a >4ch layout.

Yep, it appears these decoders with remixing capability output the full channels if they can't remix to the requested layout.

[ghost]

OK, pushed the branch I was talking about to master. --channels still has the broken old semantics, but I'm not actually sure how exactly it should work.

You can force 4.0 playback with: mpv file.mkv --af=force=channels=4.0 --channels=4

However, this will upmix even stereo to 4.0, which is perhaps not what you want, or what might be fine anyway.

Please test.

[rvalles]

Built current git (I'm using gentoo's -9999 ebuild). No good.

Trying to do 6->4:

$ mpv --af=force=channels=4.0 --channels=4 [a-S]\ Ghost\ In\ The\ Shell\ -\ Stand\ Alone\ Complex\ 2nd\ Gig\ (01-26)\ (1080p)/[a-s]_g.i.t.s._s.a.c._2ndgig-05-_those_with_a_motive_rs2[1080p_bd-rip][1ED85833].mkv Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 9: reading configurations from ~/.fonts.conf is deprecated. Playing [a-S] Ghost In The Shell - Stand Alone Complex 2nd Gig (01-26) (1080p)/[a-s]_g.i.t.s._s.a.c._2ndgig-05-_those_with_a_motive_rs2[1080p_bd-rip][1ED85833].mkv. Cache size set to 131072 KiB Cache fill: 14.75% (19791872 bytes)

Detected file format: Matroska Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 9: reading configurations from ~/.fonts.conf is deprecated. [stream] Video (+) --vid=0 'AVC-HD' (h264) [stream] Audio (+) --aid=0 --alang=eng (*) 'AAC' (aac) [stream] Audio --aid=1 --alang=jpn 'AAC' (aac) [stream] Subs --sid=0 --slang=eng 'ASS' (ass) Selected video codec: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 [lavc:h264] Selected audio codec: AAC (Advanced Audio Coding) [lavc:aac] [SWR @ 0x7fe8d7f29ec0]Input channel layout has a different number of channels than the number of used channels, ignoring layout [SWR @ 0x7fe8d7f29ec0]Input channel layout has a different number of channels than the number of used channels, ignoring layout AO: [alsa] 48000Hz 4.0 4ch s16le [SWR @ 0x7fe8d7f29ec0]Input channel layout has a different number of channels than the number of used channels, ignoring layout [SWR @ 0x7fe8d7f29ec0]Input channel layout has a different number of channels than the number of used channels, ignoring layout VO: [opengl-hq] 1920x1080 => 1920x1080 420p [fs] Assertion out->ch_count == av_get_channel_layout_nb_channels(s->out_ch_layout) failed at libswresample/rematrix.c:433 Aborted

And trying to do 2->4: $ mpv --af=force=channels=4.0 --channels 4 02-魔理沙は大変なものを盗んでいきました.ogg Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 9: reading configurations from ~/.fonts.conf is deprecated. Playing 02-魔理沙は大変なものを盗んでいきました.ogg. Cache size set to 131072 KiB Cache fill: 2.82% (3788459 bytes)

Detected file format: Ogg (libavformat) Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 9: reading configurations from ~/.fonts.conf is deprecated. [stream] Audio (+) --aid=0 '魔理沙は大変なものを盗んでいきました' (vorbis) Video: no video Selected audio codec: Vorbis [lavc:vorbis] [SWR @ 0x7f6aabb74ec0]Input channel layout has a different number of channels than the number of used channels, ignoring layout AO: [alsa] 44100Hz 4.0 4ch s16le [SWR @ 0x7f6aabb74ec0]Input channel layout has a different number of channels than the number of used channels, ignoring layout Assertion out->ch_count == av_get_channel_layout_nb_channels(s->out_ch_layout) failed at libswresample/rematrix.c:433 Aborted Parent process disappeared, exiting cache process.

[ghost]

A known bug in older libswresample versions. I'm not sure if they are going to fix this. Can you try a newer ffmpeg version?

[ghost]

To remove possible confusion: libswresample is a part of ffmpeg.

[rvalles]

Moved from 1.0.6 to 1.2.1 and also rebuilt mpv.

Now it doesn't crash. It's... doing something, but it's not as expected. It appears voice is coming from front (I think just on the left) and rear left, rather than front left+right as expected or not coming at all (as with just --channels 4)

So there's some paning going on, but it's wrong.

As for 2->4, it's not doing "room fill" (as desired), either. It seems to only come normally from the front.

[ghost]

Well, this is probably going to take a while to figure out.

there are about 4 possible sources of failure:

• the source file is incorrectly mixed, or the source file's channel layout is incorrectly identified (unlikely)
• libswresample is doing something wrong in remixing
• mpv's usage of ALSA is incorrect, messing up channel order
• ALSA is doing something wrong (I guess that's unlikely too, because you probably tested your setup with other programs)

What you could try is compiling ffmpeg with --enable-avresample, so mpv will use libavresample instead of libswresample. These two libraries are relatively similar, but there could be differences.

Note that I can't test most of this stuff directly, as I don't have a surround setup. All I can do is making ALSA downmix surround output.

[rvalles]

From the ebuild, it appears --enable-avresample is already there:

Mandatory configuration

myconf="
    --enable-gpl
    --enable-postproc
    --enable-avfilter
    --enable-avresample
    --disable-stripping
    ${myconf}"

And from qlist ffmpeg, it did build:

reimu ~ # qlist ffmpeg|grep resample /usr/include/libswresample/version.h /usr/include/libswresample/swresample.h /usr/include/libavresample/version.h /usr/include/libavresample/avresample.h /usr/share/doc/ffmpeg-1.2.1/swresample.txt.bz2 /usr/share/man/man1/libswresample.3.bz2 /usr/share/man/man1/ffmpeg-resampler.1.bz2 /usr/lib64/pkgconfig/libswresample.pc /usr/lib64/pkgconfig/libavresample.pc /usr/lib64/libswresample.so.0.17.102 /usr/lib64/libavresample.so.1.1.0 /usr/lib64/libavresample.so.1 /usr/lib64/libavresample.so /usr/lib64/libswresample.so.0 /usr/lib64/libswresample.so

Also worth noting: $ ldd /usr/bin/mpv|grep resamp libavresample.so.1 => /usr/lib64/libavresample.so.1 (0x00007f7c81574000) libswresample.so.0 => /usr/lib64/libswresample.so.0 (0x00007f7c7b290000)

And, as mpv isn't complaining like it was with ffmpeg 1.0.6, I do not know which of libav or libsw is being used.

As for your possible sources of failure:

1. No way. (fine with mpc-hc on Windows)
2. Maybe. (or libavresample, as I don't know what's it using)
3. Perhaps.
4. No. Other progs do fine. Alsa's "speaker-test -c4" matches the speakers properly, too.

Note (easy to forget) that even without surround speakers, if the soundcard has the outputs, you can just plug earphones in there... 6ch is very common with motherboards that use HDA audio. Perhaps worth a shot for debugging.

[ghost]

I tried what you suggested and used speaker-test. I noticed that ALSA calls the other two channels "rear left" and "rear right", while mpv/ffmpeg's 4.0 uses "front center" and "back center".

It appears that 4.0 can refer to two completely different speaker layouts, and the new code for ALSA confuses these (MPlayer probably gets it wrong too, because the old code treated them as equal).

You could try: mpv file.mkv --af=force=channels=quad:out-channels=4.0 --channels=4

I'll investigate further later, if this is right fixing the bug would be a one-line change,

[rvalles]

Yes! This parameter did it.

It seems correct. I cannot be sure, but rear is playing just music while front left+right includes the voice.

Stereo sources still don't expand to the 4 speakers, however. It'd be neat to have a way to also do that. As for making it a default, I'd be OK with it but as you said, perhaps some people would prefer otherwise. Note that expanding is what Windows and/or MPC-HC does for 4 speakers so it will be the expected behaviour for most.

[rvalles]

It's probably correct already, but it would help if we had samples that said "front left, front right, front mid, side left, side right, rear left, rear centre, rear right, subwoofer" and the like, for various typical arrangements. (at least 2ch, 4ch, 6ch, 8ch)

[ghost]

I fixed the ALSA issue in master. Now mpv sometestfile.mkv -af force=channels=quad -channels quad should be the correct command to get 4 channel output with ALSA.

Stereo sources still don't expand to the 4 speakers, however.

It appears libavresample/swresample leave newly added channels silent by default. The only way new audio is added to these new speakers is if other channels are removed. I'll have to ask around.

it would help if we had samples that said "front left, front right, front mid, side left, side right, rear left, rear centre, rear right, subwoofer" and the like, for various typical arrangements. (at least 2ch, 4ch, 6ch, 8ch)

I thought that too. Googling didn't really help. So I made my own: https://github.com/mpv-player/random-stuff/tree/master/speaker_test

[rvalles]

Updated again and tested. Seems ok for the trouble vids I was watching fine with the old parameters.

However, I've also tried your tests vids. Some do weird things, like 4.0 "front center" being hear-able by rear as loud as "rear center" and, for 7.1, front/mid/rear are all hearable in the rear in the same loudness. Other samples have similar issues.

I'm of course always assuming the samples themselves are correct.

[ghost]

4.0 "front center" being hear-able by rear as loud as "rear center"

I can't reproduce this. I tried: mpv speaker_test/speaker_test_4.0.avi -ao alsa -channels 6 -af force=channels=quad So the 4.0 output will be remixed as ALSA quad (note that this requires updated master with commit a39d369c254ff). The rear speakers are silent, except when "back center" is played. Shouldn't this be correct?

Or maybe I'm overlooking something. Listenting to surround with headphones is quite confusing, and surround in general is confusing too.

[rvalles]

Hm, that sounds correct now, for both tests. Seems I used my alias to play the samples and in the terminal I used, an older alias was in effect and the parameters were different. Disregard that.

I see you're using -channels 6. I wonder if that's more correct than quad with a 6ch source...

[ghost]

I see you're using -channels 6. I wonder if that's more correct than quad with a 6ch source...

I just used -channels with a random value (other than 2) to avoid stereo downmix. The -channels switch is confusing at best, and I should probably change it. I was thinking about making it behave similar to -af force=channels=..., with the exception that subset layouts like stereo are not upmixed, or something like this. Suggestions welcome!

Also, thanks for going through all the testing.

[ghost]

Actually, I'll just close this.