alsa: multichannel PCM unsupported

[ghost]

Apparently this doesn't work yet. If I interpret this correctly, there is no kernel support for this yet. If I force ALSA to output more than 2 channels, none of them seem to arrive at the receiver either (unfortunately my receiver doesn't show the PCM format it gets).

I wonder what the 8 sub devices are for?

[popcornmix]

The 8 sub devices allow up to 8 processes to output audio simultaneously (and they get mixed together by GPU).

@julianscheel didn't you get multichannel alsa working? It may be as simple as rebuilding alsa driver to report 8 channels as the maximum.

[ghost]

Note that as far as userland is concerned, proper multichannel support will also need channel map support. But the alsa-lib on Raspbian is too old and does not support this API. It will require at least alsa-lib 1.0.27. (At least mpv and vlc rely on this.)

[julianscheel]

@popcornmix I did not try PCM multichannel so far. What I was doing is being able to use HDMI and analog output in parallel from different substreams (https://github.com/raspberrypi/linux/issues/847) - I never cleaned that code up to push it, though :( - I hope I'll find some time to do it. Unfortunately I don't even have any >2 channel HDMI audio receiver, so I can't test that.

[ghost]

Is there anything I could help with? Shall I get the kernel sources, change the maximum channel count to 8, and see what happens?

[popcornmix]

Yes, please do. It might just work (apart from channel mapping). I did add support for multichannel to the firmware side a while back, but without anyone actively testing it, it wasn't enabled in kernel driver.

If you know how the channel mapping is meant to be passed from alsa userland -> alsa driver then I suspect I can support that (you can adjust channel mapping though a vcgencmd, but a standard way would be better).

[ghost]

Yes, please do. It might just work (apart from channel mapping). I did add support for multichannel to the firmware side a while back, but without anyone actively testing it, it wasn't enabled in kernel driver.

Will try later this week.

If you know how the channel mapping is meant to be passed from alsa userland -> alsa driver then I suspect I can support that (you can adjust channel mapping though a vcgencmd, but a standard way would be better).

Not yet, but maybe I can try to understand it.

[ghost]

Raising the max. channel count to 8 does work. I can output 6 channels, and they all arrive separately on the receiver. 8 channels also appears to work, except I can't test this properly because my receiver can do 5.1 only.

The only problem is that the ALSA API still doesn't report any channel maps, so this part still needs to be implemented. Haven't looked into it.

DTS-HD passthrough also almost works. I just had to raise the max. channel count to 8, and the max. samplerate to 192000 in the spdif device.

[ghost]

I've taken a small look. Channel maps are an additional control element (like the IEC958 switch), which is added with snd_pcm_add_chmap_ctls(). It seems there are basically two uses of this function:

• static list of channel maps (channel count uniquely identifies the channel layout, at least I guess)
• dynamic usage (callback fun)

The HDA HDMI driver (https://github.com/raspberrypi/linux/blob/rpi-3.18.y/sound/pci/hda/patch_hdmi.c#L2132) and the generic USB driver (https://github.com/raspberrypi/linux/blob/rpi-3.18.y/sound/usb/stream.c#L217) show somewhat how to use channel maps dynamically. They also set up a bunch of callbacks on the new control (get/set/tlv.c). I have no idea whether the API actually requires using callbacks to get dynamic behavior with multiple possible layouts per channel count. The callback stuff really looks pretty ugly (letting a driver manually set up really weird userspace data structures...?)

The HDA HDMI driver also lists a bunch of HDMI layouts, which might be useful: https://github.com/raspberrypi/linux/blob/rpi-3.18.y/sound/pci/hda/patch_hdmi.c#L278 https://github.com/raspberrypi/linux/blob/rpi-3.18.y/sound/pci/hda/patch_hdmi.c#L776 (plus some more mapping code spread over the file)

Also, I don't know if HDMI can report the channel layouts supported by the device via EDID or so. Seems not?

On the RPI side, the only way I've found to set the channel layout is via sending the hdmi_channel_map command to the GCMD service. This is what xbmc does (uses vc_gencmd()). I haven't found anything related in vc_vchi_audioserv_defs.h. This sounds annoying, you'd have to create extra vchi connections and all. Maybe the firmware actually provides a better way? (The firmware is still closed source.)

If nobody else does, I might attempt to give it a try.

[popcornmix]

DTS-HD passthrough also almost works. I just had to raise the max. channel count to 8, and the max. samplerate to 192000 in the spdif device.

Are you saying you have had HD audio successfully play through HDMI on Pi? I can play 192kHz 8 channel PCM but I believe we drop samples at that rate. Any attempt at passthrough fails to lock on. (Although there was a firmware tweak to the fifo settings a couple of weeks back that could have helped).

[popcornmix]

If nobody else does, I might attempt to give it a try.

If you can plumb in the support into kernel driver, then I can problably help on the firmware side. I could add a new method to audioservice (e.g. VC_AUDIO_MSG_TYPE_CHANNELMAP) that takes a parameter like the vcgencmd hdmi_channel_map.

In theory the edid can be read (tvservice -d edid.dat) and the firmware does parse this so could provide it through other means. I believe that in general edid information is not considered reliable enough to used by default. We used to automatically choose passthrough options in Kodi based on edid, but have since made this purely a user selectable option as there are just too many bad edids out there (Kodi doesn't trust the edid on windows/linux platforms either).

[ghost]

Are you saying you have had HD audio successfully play through HDMI on Pi? I can play 192kHz 8 channel PCM but I believe we drop samples at that rate.

I can try again and pay attention to the quality.

Any attempt at passthrough fails to lock on.

The receiver didn't seem to complain when playing DTS-HD.

If you can plumb in the support into kernel driver,

I can try it. No guarantee for success. It'd be my first time hacking on a Linux driver.

then I can problably help on the firmware side. I could add a new method to audioservice (e.g. VC_AUDIO_MSG_TYPE_CHANNELMAP) that takes a parameter like the vcgencmd hdmi_channel_map.

Sounds ideal.

In theory the edid can be read (tvservice -d edid.dat) and the firmware does parse this so could provide it through other means. I believe that in general edid information is not considered reliable enough to used by default. We used to automatically choose passthrough options in Kodi based on edid, but have since made this purely a user selectable option as there are just too many bad edids out there (Kodi doesn't trust the edid on windows/linux platforms either).

Then it's probably fine not to care about this aspect.

[popcornmix]

@wm4 Let me know how you were playing the file (perhaps point at the file you used and commands used to play it) and I'll see if I get the same results here (I've got a Yahama receiver at work and an Onkyo at home).

Before the recent fifo tweaks we found some receivers played stereo 192kHz audio perfectly, but some suffered from dropouts. You might imagine either we are outputting valid data or we weren't but it seems the real answer is more complicated.

What receiver are you using? It possibly is more tolerant than mine.

[ghost]

What receiver are you using?

Pioneer VSX-329-K

Also I haven't tested anything with 192khz PCM, only DTS-HD (which pretends 192khz via spdif to reach the required bitrate). The rest I'll test later.

[ghost]

I tried 192Khz PCM. Yes, it seems to run into problems here. I tried to play a 192khz 8ch float wav file, and it seems mpv isn't fast enough to push the data to ALSA. It also uses too much CPU. (Probably too much memcpy.) aplay seems to work fine though. When using s16 instead of float it's better (alsalib doesn't need to convert anymore), then aplay barely uses CPU and I can't recognize any dropouts.

I actually can't get DTS-HD to work. The receiver displays the expected text on the OSD (which indicates DTA MA), but there's only silence, and then some clicks. I'm not sure what happens. I've tried to make a wav to get aplay to stream DTS-HD to the receiver, but this didn't work. (I guess the audio got mangled somewhere along the way, or I didn't set the mixer to 0db correctly.) I'm not sure if it actually worked when I tested it last time, or if I only looked at the receiver display.

[popcornmix]

I've never got DTS-HD passthrough working. I'd be very interested if you do actually get audio out, but I suspect it may not be possible. Whilst an 8channel 192kHz wav file can almost be played, I suspect if you listen carefully to a sine wave you will hear the odd glitch. The hardware wasn't really designed for this data rate. I believe 192kHz stereo is perfect and 48kHz 8 channel is perfect, but 192kHz 8 channel does drop the odd sample.

[ghost]

I've never got DTS-HD passthrough working. I'd be very interested if you do actually get audio out, but I suspect it may not be possible.

Other than the receiver indicating DTS-MA, no. So if you're right and 192khz @ 8ch subtly drops or changes samples, then it could be that the receiver gets the correct DTS headers, but is then unable to decode the DTS frame correctly. The result is silence, or the occasional clicks I was hearing - but the receiver won't try to interpret it as PCM, because the next valid DTS header and/or SPDIF frame is not far. Makes sense to me.

Whilst an 8channel 192kHz wav file can almost be played, I suspect if you listen carefully to a sine wave you will hear the odd glitch.

My hearing isn't particularly good. Any tips how to generate the sine wave? (Or rather, what frequency would be good.)

The hardware wasn't really designed for this data rate. I believe 192kHz stereo is perfect and 48kHz 8 channel is perfect, but 192kHz 8 channel does drop the odd sample.

Does this restriction definitely apply to kodi's OMX output as well? (It certainly should, since it's the same hw, I'm just asking for definite confirmation so I don't need to possibly pursue an OMX solution.)

Maybe there's no other choice but to restrict output this way. I guess I'll need to find out how to do this with the ALSA kernel API. What is the actual hw limit based on? On a maximum bit rate?

[popcornmix]

/opt/vc/src/hello_pi/hello_audio will generate a sine wave using openmax and you can specify channels and samplerate on command line.

The limit is on the GPU peripheral (I believe the audio fifo in the hdmi block), so omx and alsa will behave the same.

Actually just checking but we currently limit channels_samplerate to 8_96000. If you output more than this the GPU will resample then audio down to below this limit (e.g. 8 channel 192kHz sudio will be resampled to 8 channel 96kHz). You can adjust this with hdmi_samples_limit in config.txt. E.g. hdmi_samples_limit=1536000 should bypass this limit. By default it is 768000. So I believe that would have made DTS-HD passthrough impossible. Might be worth trying again with the increased limit (but I'd be surprised if it works).

[ghost]

I tried with hdmi_samples_limit=1536000 in config.txt. It doesn't seem to help too much. DTS-HD is broken in the same way (although, if it really had been resampled with default settings, I can't explain how the DTS header made it to the receiver). hello_audio 1 8 192000 plays a tinnitus-inducing sound, but seems to be ok. I can't make out any distortions.

Well, 8 channels at 96khz is still great, and having yet another reason to discourage DTS passthrough is actually a feature in my book. (I just hope libdcadec is fast enough.)

[popcornmix]

We support libdcadec in kodi and it's fine on Pi2. On Pi1 it's a bit tight - about half the speed of ffmpeg's dca decoder.

[popcornmix]

Your receiver probably has an info/display mode that shows the samplerate and number of channels. That should confirm if 192kHz is being output or if it's being resampled down. Note: If edid doesn't report support for a frequency we will resample down. Check with tvservice -a

[ghost]

Unfortunately the receiver doesn't seem to display this anywhere.

I'll check with tvservice tomorrow.

[amtssp]

Sorry for asking the obivious question. Are you sure that alsa does not change output level? It should be set to 100% otherwise it Will not output bit perfect.

[ghost]

tvservice -a:

     PCM supported: Max channels: 8, Max samplerate: 192kHz, Max samplesize 24 bits.
     AC3 supported: Max channels: 6, Max samplerate:  48kHz, Max rate  640 kb/s.
     DTS supported: Max channels: 7, Max samplerate:  48kHz, Max rate 1536 kb/s.
  DTS_HD supported: Max channels: 8, Max samplerate: 192kHz, Max rate    8 kb/s.

(On a side note, does RPI support 24 bit sample output?)

@amtssp: normal DTS output works just fine.

[popcornmix]

Yes, 24-bit samples are supported by hardware.

[pssc]

But this is not enabled in the alsa driver... (~3.18) sound/arm/bcm2835-pcm.c

 static struct snd_pcm_hardware snd_bcm2835_playback_hw = {
        .info = (SNDRV_PCM_INFO_INTERLEAVED | SNDRV_PCM_INFO_BLOCK_TRANSFER |
                 SNDRV_PCM_INFO_MMAP | SNDRV_PCM_INFO_MMAP_VALID),
-       .formats = SNDRV_PCM_FMTBIT_U8 | SNDRV_PCM_FMTBIT_S16_LE,
-       .rates = SNDRV_PCM_RATE_CONTINUOUS | SNDRV_PCM_RATE_8000_48000,
+       .formats = SNDRV_PCM_FMTBIT_U8 | SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S24_LE,
+       .rates = SNDRV_PCM_RATE_CONTINUOUS | SNDRV_PCM_RATE_8000_192000,
        .rate_min = 8000,
-       .rate_max = 48000,
+       .rate_max = 192000,

[ghost]

Didn't try to enable S24 yet.

For 192khz output, I added something to my private branch which restricts it to channel counts which the RPI supposedly can handle without dropouts.

[pssc]

yeah I haven’t been able to test this yet on my private build due to device tree issues, but 24bit 96k would be of use event at 2 channel this would be well within the pi's caps?

[popcornmix]

@wm4 you shouldn't have to restrict channels x samplerate as the firmware should downsample what it thinks is unsupported anyway.

In theory 192kHz, 8 channel and 24bit (or 32bit if useful with GPU discards 8 lsb) should work but are untested. If you confirm what does work, we'll enable in ALSA driver. If you point out what doesn't work, we can probably support it in firmware.

[ghost]

@wm4 you shouldn't have to restrict channels x samplerate as the firmware should downsample what it thinks is unsupported anyway.

I don't know, personally I'd prefer if it didn't resample to a lower samplerate. Actually, maybe it shouldn't resample at all. The downside is that resampling is almost certainly more efficient on the GPU, even though it uses an unknown algorithm with unknown implementation quality.

[popcornmix]

I believe the gpu resampling will be higher quality than anything done on the arm. And the arm is not going to handle resampling 8 channel 192kHz audio on a Pi1 (and I'm not sure about Pi2).

[julianscheel]

Maybe we could create different audio devices? One that uses the resampling and thus reports up to 192kHz and another one that disables any resampling on the GPU and reports only supported rates? This would allow users to have the full control while still being able to make use of the GPU features...

[pssc]

S24 produced noise plus high level noise will experiment with S24U 3B tomorrow , we just need alsa device that translates downwards to what the device for tv service reports... both in terms or sample size / type and rate...

[pssc]

SNDRV_PCM_FMTBIT_S24_LE | SNDRV_PCM_FORMAT_S24_3LE @ 96k and 192k now I will drop the sample rate to force software re-sampling alsa side to 48k-- this also fails

[pssc]

ok and I cant confirm that no sample formats work so far all produce noise with some level of audio regardless of sample rate S32LE tested @ 96k 196k and 48k

[popcornmix]

Can you let me know the change you made to alsa driver and which commands to run to see any problems with 24 or 32-bit audio.

[ghost]

OK, here is it: https://github.com/wm4/linux/tree/multichannel_pcm

There's probably still much wrong with it. It's full of debug log calls with too high severity (error for mere debug messages). Locking probably isn't done correctly (but I couldn't see any locking for the volume control either).

Also, I have this issue: when playing quad (FL FR RL RR channel layout), my receiver plays the RL channel on both surround left/right speakers, while the RR channel is never heard (or maybe it was the other way around). The videocore channel map is 0x08000b08, which is CEA 0x8 (which is correct), a channel count of 4, and the following mapping:

[ 3211.207512] bcm2835_audio_set_ctls_chan:585 remap 0 -> 0
[ 3211.207522] bcm2835_audio_set_ctls_chan:585 remap 1 -> 1
[ 3211.207531] bcm2835_audio_set_ctls_chan:585 remap 2 -> 4
[ 3211.207539] bcm2835_audio_set_ctls_chan:585 remap 3 -> 5
[ 3211.207548] bcm2835_audio_set_ctls_chan:585 remap 4 -> 0
[ 3211.207556] bcm2835_audio_set_ctls_chan:585 remap 5 -> 0
[ 3211.207565] bcm2835_audio_set_ctls_chan:585 remap 6 -> 0
[ 3211.207574] bcm2835_audio_set_ctls_chan:585 remap 7 -> 0

which also looks correct to me. What's wrong?

hello_audio.bin 1 4 without changing the channel map set in this way plays audio on both surround channels.

[pssc]

diff --git a/sound/arm/bcm2835-pcm.c b/sound/arm/bcm2835-pcm.c
index 8c86375..0f25681 100755
--- a/sound/arm/bcm2835-pcm.c
+++ b/sound/arm/bcm2835-pcm.c
@@ -23,9 +23,12 @@
 static struct snd_pcm_hardware snd_bcm2835_playback_hw = {
    .info = (SNDRV_PCM_INFO_INTERLEAVED | SNDRV_PCM_INFO_BLOCK_TRANSFER |
         SNDRV_PCM_INFO_MMAP | SNDRV_PCM_INFO_MMAP_VALID),
-   .formats = SNDRV_PCM_FMTBIT_U8 | SNDRV_PCM_FMTBIT_S16_LE,
+// .formats = SNDRV_PCM_FMTBIT_U8 | SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S24_LE | SNDRV_PCM_FORMAT_S24_3LE,
+   .formats = SNDRV_PCM_FMTBIT_U8 | SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S24_LE | SNDRV_PCM_FORMAT_S24_3LE | SNDRV_PCM_FMTBIT_S32_LE,
+// .rates = SNDRV_PCM_RATE_CONTINUOUS | SNDRV_PCM_RATE_8000_192000,
    .rates = SNDRV_PCM_RATE_CONTINUOUS | SNDRV_PCM_RATE_8000_48000,
    .rate_min = 8000,
+// .rate_max = 192000,
    .rate_max = 48000,
    .channels_min = 1,
    .channels_max = 2,

Selection Relevant options in code here and simple testing with files from http://www.naimlabel.com/musicstore-test-files.aspx aplay -v naim-test-2-wav-24-96000.wav

[pssc]

@popcornmix what algorithm does the VC core implement for resampling? its probably speex(good) or linear-interpolation (in which case HQ audio applications will want to by pass it)

[ghost]

@popcornmix what algorithm does the VC core implement for resampling? its probably speex(good) or linear-interpolation (in which case HQ audio applications will want to by pass it)

Well, it seems natural that you want to bypass as much as proprietary shit processing as possible. "Promises" don't help much.

[pelwell]

@wm4 Please adjust your attitude. When @popcornmix wrote:

I believe the gpu resampling will be higher quality than anything done on the arm. And the arm is not going to handle resampling 8 channel 192kHz audio on a Pi1 (and I'm not sure about Pi2).

it wasn't an idle boast. The vector processor at the heart of VideoCore is extremely capable, and allows the bcm2835 to punch above its weight, and the audio resampling software is of a very high standard.

Although closed source is bad, mmmkay, don't make the mistake of assuming closed source software is bad software.

[ghost]

Although closed source is bad, mmmkay, don't make the mistake of assuming closed source software is bad software.

Well, from what I've seen from the outside and from the open sourced userspace parts, the broadcom code is a quite hilarious piece of software. If I were an audiophile I literally would not trust it one bit.

[JamesH65]

Hmm, with literally millions of devices using this exact code sold to people like Roku, Samsung, Amazon for use in video and audio circumstances, and given that the people who wrote a lot of that code are the very people you are talking too, I also think you need to adjust your attitude. You have not seen the closed source code. Therefore you are unable to comment on it.

On 11 June 2015 at 10:32, V. Lang notifications@github.com wrote:

Although closed source is bad, mmmkay, don't make the mistake of assuming closed source software is bad software.

Well, from what I've seen from the outside and from the open sourced userspace parts, the broadcom code is a quite hilarious piece of software. If I were an audiophile I literally would not trust it one bit.

— Reply to this email directly or view it on GitHub https://github.com/raspberrypi/linux/issues/1000#issuecomment-111063342.

[ghost]

You have not seen the closed source code. Therefore you are unable to comment on it.

Exactly. I'm not sure why you blame me for not trusting it. Anyway, this is not even important. You do admit that the firmware has all sorts of quirkses which I cannot even attempt to fix, right? And that it requires the use of under-documented proprietary APIs etc.? I can adjust my attitude not to point out these facts - I guess it ruins the fun people are having with these devices, thus is completely unwelcome. Of course, it's all the same with SoCs in general and is not necessarily RPI specific.

Still, I think a driver shouldn't pretend to support more than the hardware actually can do. Exposing separate devices for "pure hardware" and "goes through videocore processing" as it was suggested earlier might actually be a good idea.

(Going back to my chmap code and trying to figure out what exactly the firmware is doing.)

[haasn]

and the audio resampling software is of a very high standard.

What algorithm does it implement?

[JamesH65]

I'm blaming you for commenting on the quality of source YOU HAVE NOT SEEN, code that is used by some of the biggest suppliers of equipment out there, who all have high standards(yes, they actually do, I've been on the receiving end of some of it), and are all using it with no problem. Nothing else.

As to what the algorithm is, dunno. It's a black box - built by a professional bunch of engineers who know what they are doing, it's works, therefore I'm happy to use it and trust it.

On 11 June 2015 at 10:52, V. Lang notifications@github.com wrote:

You have not seen the closed source code. Therefore you are unable to comment on it.

Exactly. I'm not sure why you blame me for not trusting it. Anyway, this is not even important. You do admit that the firmware has all sorts of quirkses which I cannot even attempt to fix, right? And that it requires the use of under-documented proprietary APIs etc.? I can adjust my attitude not to point out these facts - I guess it ruins the fun people are having with these devices, thus is completely unwelcome. Of course, it's all the same with SoCs in general and is not necessarily RPI specific.

Still, I think a driver shouldn't pretend to support more than the hardware actually can do. Exposing separate devices for "pure hardware" and "goes through videocore processing" as it was suggested earlier might actually be a good idea.

(Going back to my chmap code and trying to figure out what exactly the firmware is doing.)

— Reply to this email directly or view it on GitHub https://github.com/raspberrypi/linux/issues/1000#issuecomment-111071124.

[julianscheel]

@wm4 I don't think you were blamed for not trusting it. But stating things like

proprietary shit processing

or

the broadcom code is a quite hilarious piece of software

is not really about not trusting but quite a harsh blame.

I fully understand that you'd like to know the technical details and/or have a way to disable it completely, but a more polite wording might help here...

[haasn]

As to what the algorithm is, dunno. It's a black box - built by a professional bunch of engineers who know what they are doing, it's works, therefore I'm happy to use it and trust it.

Black boxes can be analyzed, there's no need to make assumptions. Has anybody measured the output of the resampling process and compared it to known references eg. SoX or src-sinc-best?

[ghost]

the broadcom code is a quite hilarious piece of software

Sorry about that. The fact that you could quite easily freeze or break videocore completely until the next reboot just by through incorrect API usage from userspace made me not trust this device.

[pelwell]

Since you all asked so nicely, I logged in on my day off to look at the algorithm. I didn't do engineering, and I my DSP experience is limited, but the comments link to this page:

https://ccrma.stanford.edu/~jos/resample/Implementation.html

From what I can tell, it is using 512 samples per zero-crossing ("which is what we use at CCRMA, and which is somewhat over designed") and a total filter length of 15360.

If you are still concerned about the quality, try listening to it. Find some audio that would require rate conversion, use the best available resampler to generate a version that doesn't require rate conversion, and play both files through the Pi for an A/B comparison. If you want to do a blind test (and you should), write a script that chooses the order at random. I don't think you will hear any difference.

[JamesH65]

Make sure you use OFC cables, preferably from Monster.

On 11 June 2015 at 13:43, Phil Elwell notifications@github.com wrote:

Since you all asked so nicely, I logged in on my day off to look at the algorithm. I didn't do engineering, and I my DSP experience is limited, but the comments link to this page:

https://ccrma.stanford.edu/~jos/resample/Implementation.html

From what I can tell, it is using 512 samples per zero-crossing ("which is what we use at CCRMA, and which is somewhat over designed") and a total filter length of 15360.

If you are still concerned about the quality, try listening to it. Find some audio that would require rate conversion, use the best available resampler to generate a version that doesn't require rate conversion, and play both files through the Pi for an A/B comparison. If you want to do a blind test (and you should), write a script that chooses the order at random. I don't think you will hear any difference.

— Reply to this email directly or view it on GitHub https://github.com/raspberrypi/linux/issues/1000#issuecomment-111119887.