search

respeaker / seeed-voicecard

2 Mic Hat, 4 Mic Array, 6-Mic Circular Array Kit, and 4-Mic Linear Array Kit for Raspberry Pi
GNU General Public License v3.0
487 stars 294 forks source link

6mics / linear 4 mics: fails to record against v5.10 kernel #290

Closed HinTak closed 3 years ago

HinTak commented 3 years ago

See two people (cc @Drizzt321 @BeckyHeath ) experience such problem, probably should start a new issue instead of keeping on exchange elsewhere (#285), especially on closed issues (#234)

lxne commented 3 years ago

Same issue. Not working on my Pi 4 with most recent buster and kernel v5.10.

flattman commented 3 years ago

Hey, i have the same problem with my 6 mic array. I can play a wav file but can not record. It only produces empty wave files with this cmd arecord -Dac108 -f S32_LE -r 16000 -c 8 a.wav, same in audacity.

I have installed a fresh raspbian and only installed the driver (v5.9).

my system: Linux raspberrypi 5.10.17-v7l+ #1403 SMP Mon Feb 22 11:33:35 GMT 2021 armv7l GNU/Linux Raspbian GNU/Linux 10 (buster) Raspberry Pi 4 Model B Rev 1.2

i don't know if it is relevant, but i can see some "I2S SYNC error!" and "overlay: WARNING: memory leak will occur if overlay removed" in my dmesg. I can play with the leds, so i guess my cable is connected correctly.

lxne commented 3 years ago

I just figured out that for my Pi 4 and 4 mic linear array the respeaker/seeed-voicecard/master with '--compat-kernel'-option works.

Drizzt321 commented 3 years ago

@xiongyihui yes, except that downgrades the kernel to an older version.

HinTak commented 3 years ago

Raspbian's kernel upgrade is a lot more drastic than desktop Linux's (jumping from 4.10 to 4.19 to 5.4 to 5.10...) . I haven't even updated my Ubuntu pi from 5.4 to 5.8 yet... I think I'll go via 5.8 to see whether it breaks between 5.4 and 5.8 or 5.8 to 5.10... It is going to take a while - also because I now have all 3 devices...

HinTak commented 3 years ago

Yes, the driver broke between 5.4 and 5.8 . arecord blocking at snd_pcm_mmap_readi().

Edit: only the 6-mic (and the linear 4-mic) is not recording at 5.8. The 2-mic and square 4-mics okay at 5.8.

HinTak commented 3 years ago

@Drizzt321 @BeckyHeath @lxne @flattman @xiongyihui : which devices have you got? I found it is only the 6mics (and linear 4-mics) broke at 5.8 . 2-mics and square 4-mics are still okay at 5.8 .

disclaimer: I don't work for seeed studio - though my 2-mics and square 4-mics came from a gift-voucher courtesy of seeed studio.

HinTak commented 3 years ago

I have finished testing all three devices (2 and 6 mics and the square 4-mics) I have against a fresh new current rasp os (which ships with 5.10) and just the 6-mics - and likely the linear 4-mics - is broken. Have enabled debugging and started switching back and forth between 5.10 and 5.4 with the 6-mics and it is a bit slow, but at least we know the problem is probably in the ac101-part.

I don't particularly want to encourage people to downgrade, but for the time being I am switching back and forth between current 5.10.x and last 5.4.x to find out what's wrong, and I have written a how-to. https://github.com/HinTak/RaspberryPi-Dev/blob/master/Downgrading-Pi-Kernel.md . People can go backward and hold, until the issue is fixed, I guess.

Edit : correcting url

lxne commented 3 years ago

Hi @HinTak, I do have the linear 4-mic. Thank you so much for your investigations. Your link to the how-to switch sends me to 'not found'. Am I missing something?

BeckyHeath commented 3 years ago

@HinTak Howdy! I have the 6-mic setup. I ended up solving the issue by running an old version of raspbian buster instead. see: https://downloads.raspberrypi.org/raspbian_full/images/raspbian_full-2020-02-14/

flattman commented 3 years ago

@HinTak thxs for your investications. I can confirm, that my 6-array mic works like a charm with kernel 5.4, installed by --compat-kernel. I guess that for raspberry os there are only 5.4 and 5.10 kernels, right?

HinTak commented 3 years ago

@lxne sorry, corrected the url https://github.com/HinTak/RaspberryPi-Dev/blob/master/Downgrading-Pi-Kernel.md

@BeckyHeath - good to know the full version have matching kernel header package included. Starting with lite/headless, the matching header package usually missing, and it is not always possible to get the header without upgrading the kernel.

HinTak commented 3 years ago

@flattman you likely want to do the "apt-mark hold..." step too (and unhold in the future). I went to the raspberrypi site, debian buster with 4.19 is also linked from there. It seems that the raspberry pi foundation is jumping from one kernel LTS version to the next. (which is 5.10, 5.4, 4.19, 4.10, etc).

HinTak commented 3 years ago

Btw, Ubuntu groovy is using kernel 5.8.x .

HinTak commented 3 years ago

Comparing the boot message under 5.4 and 5.10, one line is missing:

seeed-voicecard soc:sound: ac10x-codec.1-0035 <-> fe203000.i2s mapping ok

So it looks like the overlay is not loaded/read properly. This makes sense as the overlay is specific to 6-mics/linear-4-mics, so only the 6-mics/linear-4-mics breaks. We have had an overlay-related problem (and fixed in https://github.com/respeaker/seeed-voicecard/commit/c4c112dcafbc83ce19730bf53429a3b39bd38ca3 ) not-too-distant in the past - mostly in upstream tightening / changing what is regarded as "valid". This is probably going to be a similar issue...

HinTak commented 3 years ago

The missing "mapping ok" message turned out to be a red herring: the Intel Soundwire guy decided it is too much information (apparently each Intel audio hardware gives multiple lines of "mapping ok"...) so he decided to delete it for suppress it for everybody... Sigh. Need to looking elsewhere.

HinTak commented 3 years ago

Filed uptream as https://github.com/raspberrypi/linux/issues/4279 - pretty sure it is due to some subtle changes in the rest of the kernel...

@lxne - the red herring's change message - it seems that some Intel Soundwire device put out 14 lines of those, and this guy decided that it is not useful for the rest of us :-( : 

commit 1d5cd5254f67bc65622f4cac04c25d6e082f21b0
Author: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Date:   Fri Jun 12 15:40:50 2020 -0500

    ASoC: soc-pcm/compress: reduce verbosity on mapping ok messages

    With dynamic debug not enabled, we still get these messages:

    [   48.133586] sof_sdw sof_sdw: rt711-aif1 <-> SDW0 Pin2 mapping ok
    [   48.133595] sof_sdw sof_sdw: rt711-aif1 <-> SDW0 Pin3 mapping ok
    [   48.133650] sof_sdw sof_sdw: sdw:1:25d:1308:0 <-> SDW1 Pin2 mapping ok
    [   48.133658] sof_sdw sof_sdw: rt715-aif2 <-> SDW3 Pin2 mapping ok
    [   48.133666] sof_sdw sof_sdw: intel-hdmi-hifi1 <-> iDisp1 Pin mapping ok
    [   48.133672] sof_sdw sof_sdw: intel-hdmi-hifi2 <-> iDisp2 Pin mapping ok
    [   48.133677] sof_sdw sof_sdw: intel-hdmi-hifi3 <-> iDisp3 Pin mapping ok
    [   48.133712] sof_sdw sof_sdw: snd-soc-dummy-dai <-> Headphone 0 mapping ok
    [   48.133733] sof_sdw sof_sdw: snd-soc-dummy-dai <-> Headset mic 1 mapping ok
    [   48.133746] sof_sdw sof_sdw: snd-soc-dummy-dai <-> SDW1-speakers 2 mapping ok
    [   48.133762] sof_sdw sof_sdw: snd-soc-dummy-dai <-> Microphones 4 mapping ok
    [   48.133774] sof_sdw sof_sdw: snd-soc-dummy-dai <-> HDMI1 5 mapping ok
    [   48.133798] sof_sdw sof_sdw: snd-soc-dummy-dai <-> HDMI2 6 mapping ok
    [   48.133809] sof_sdw sof_sdw: snd-soc-dummy-dai <-> HDMI3 7 mapping ok

    This is not really useful for most users, demote to dev_dbg()
HinTak commented 3 years ago

Written to alsa-devel https://mailman.alsa-project.org/pipermail/alsa-devel/2021-April/183770.html and CC'ed the 5 people whose kernel code changes are involved. Let's see what follows on alsa-devel . More detail upstream at https://github.com/raspberrypi/linux/issues/4279 , but this is now a further upstream issue, with the rest of the linux kernel sound system, not specific to pi... Holy sh*t...

HinTak commented 3 years ago

The problematic commit is this one. I have flipped it back and forth to make sure. Unfortunately the ti.com address bounced, so I hope some of the other alsa-devel people can help.

commit 4378f1fbe924054a09ff0d4e39e1a581b9245252
Author: Peter Ujfalusi <peter.ujfalusi@ti.com>
Date:   Fri Sep 27 10:16:46 2019 +0300

    ASoC: soc-pcm: Use different sequence for start/stop trigger
HinTak commented 3 years ago

I think the upstream change was correct - that the respeaker code depends on the previous behavior is wrong... For now I have spent enough time on it and would rather doing something else for a bit. :-)

plbossart commented 3 years ago

The missing "mapping ok" message turned out to be a red herring: the Intel Soundwire guy decided it is too much information (apparently each Intel audio hardware gives multiple lines of "mapping ok"...) so he decided to delete it for suppress it for everybody... Sigh. Need to looking elsewhere.

You will see the log if you enable dynamic debug.

HinTak commented 3 years ago

@plbossart apologies -I was being unkind, after chasing down that path for a day. FWIW, I filed an issue for raspbian to ship their kernels with dynamic debugging on a year ago: https://github.com/raspberrypi/linux/issues/3486 . Most desktop Linux have it on; I kind-of understand that they want to ship with the best performance on a small ARM-based distro though.

HinTak commented 3 years ago

I have an ugly-but-functional change to the driver code to fix this, posted as https://mailman.alsa-project.org/pipermail/alsa-devel/2021-April/184023.html - anybody knowledgeable in audio drivers, advices and suggestions on how to achieve the same result in a better way would be appreciated.

HinTak commented 3 years ago

I found another way of fixing the issue I am happier with https://github.com/HinTak/seeed-voicecard/commit/19067f3333d7759bdf626e62bc09a7deae4627a1 (the first was a couple of commits before that, and reverted). @Drizzt321 @BeckyHeath @flattman @lxne @xiongyihui

I am somewhat against the --compat-kernel option (hence removed it), but if you want to downgrade the kernel, here is how to do it properly (and undo): https://github.com/HinTak/RaspberryPi-Dev/blob/master/Downgrading-Pi-Kernel.md In the course of looking at this, I have gone back and forth between 5.4 and 5.10 (and occasionally even 4.19), sometimes a few times a day so anything between 5.4 and 5.10 (including Ubuntu's 5.8) should work; and built and booted custom built kernels... There are a few in-progress write-ups, Building-the-Linux-Kernel.md , Debugging-Regressions.md Unload-Reloading-Part-2.md which hopefully will get updated before I forget all the details.

The current https://github.com/HinTak/seeed-voicecard/ works for anything between 4.19, 5.4 and 5.10. I have definitely spent way too much time on this than I am willing to... Please feel free to donate at the bottom of the page at https://hintak.github.io/ .

cc many people cloned/starred my repo recently : @foolishcui @sveinbjornt @locomoco28 @dony71 @stdevel @dbelca @Lederhaut @tcgumus @coxtor @njhurst @BeneyKim @tewarid @datagutt @Daenara

EDIT: adjust the default branch to work with 4.19 too.

Drizzt321 commented 3 years ago

@HinTak This all seems to work just fine on my fresh install w/5.10! Thanks for all the work!

It only seems to support S32_LE playback, for some reason though. Not sure what's up with that. Trying to figure out how to get aplay/arecord/etc to use that by default.

And along with the S32_LE thing, need to figure out how to get Picroft/Mycroft to use the right output, or right format for output, so I can get it working...

HinTak commented 3 years ago

@Drizzt321 aplay -L gives you a list of usable logical devices. ac101 and hw:... interacts directly with the hardware and is limited by what the hardware permits. If you want rate conversions etc, use the plugin... ones; these are defined in the asound.conf, and you can make up your own config too, to get alsa (the upper closer to application part) to do the conversion for you.

Btw, although playback takes 8 channels, a random stereo pair is used, so if you playback what you just record, there is a 1/4 chance in the 6-mics or 1/2 chance for the 4-mics you'd be playing the empty channel's :-). This is mentioned at the bottom of the Readme.

Anyway, I really would rather be doing something not respeaker related for a while :-). I don't work for Seeed Studio... if you feel generous, please click the donate link in https://hintak.github.io , even if only to send a beer!

Drizzt321 commented 3 years ago

@HinTak I think I can manage to muddle through now, now that you got the basic underlying driver worked out! And already sent you a beer, or two, depending on how much things are where you're at. :)

That's interesting, about the playback.

HinTak commented 3 years ago

@Drizzt321 thanks! You didn't have your name in github... Beer from supermarkets has got cheaper and more varied lately, since pubs are closed and breweries would rather sell some cheaply than not at all...

Anyway, the playback thing is somewhat central/relevant to this bug: recording uses the clock on the playback chip for synchronization, and when used in the designed way, data is 8-channel and same frequency in both directions; recording have two empty channels, while playback just pick a stereo pair out of 8 (hence what the readme says).

Drizzt321 commented 3 years ago

@HinTak Hm...

arecord -r 16000 test2.wav and then aplay test2.wav, and I hear it out of the mono speaker I have plugged in to the mono JST jack.

When I run the mycroft-mic-test, it doesn't tell me what the record command/values are, but it does share the playback aplay -Dhw:0,0 WAV_FILE. It then fails, saying sample format not available.

When I play it back manually, aplay /tmp/test.wav it just gives me silence, I presume because it's playing it out the wrong channels. When I specify the hardware aplay -D "plughw:0,0" /tmp/test.wav I hear it fine.

This seems quite relevant, I'll poke at it sometime soon to see if that helps me fix it. https://community.mycroft.ai/t/picroft-with-i2s-audio/4658/16

Here's the detailed mycroft output. 

$ mycroft-mic-test -l
Stopping audio (22128)...stopped.
Stopping speech (22151)...stopped.
Already up to date.
Initializing...
Starting audiotest
 Initializing...
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.front
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround40
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround41
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround50
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround51
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround71
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm_oss.c:377:(_snd_pcm_oss_open) Unknown field port
ALSA lib pcm_oss.c:377:(_snd_pcm_oss_open) Unknown field port
ALSA lib pcm_a52.c:823:(_snd_pcm_a52_open) a52 is only for playback
ALSA lib conf.c:5014:(snd_config_expand) Unknown parameters {AES0 0x6 AES1 0x82 AES2 0x0 AES3 0x2  CARD 0}
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM iec958:{AES0 0x6 AES1 0x82 AES2 0x0 AES3 0x2  CARD 0}
ALSA lib pcm_usb_stream.c:486:(_snd_pcm_usb_stream_open) Invalid type for card
ALSA lib pcm_usb_stream.c:486:(_snd_pcm_usb_stream_open) Invalid type for card
ALSA lib pcm_params.c:2226:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_params.c:2226:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_params.c:2226:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_params.c:2226:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_dmix.c:1043:(snd_pcm_dmix_open) The dmix plugin supports only playback stream
ALSA lib pcm_params.c:2226:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_params.c:2226:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_params.c:2226:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_params.c:2226:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_params.c:2226:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_params.c:2226:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_params.c:2226:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_params.c:2226:(snd1_pcm_hw_refine_slave) Slave PCM not usable
 ====================== Audio Devices ======================
  Index    Device Name
   2:       pulse
   6:       default

 ========================== Info ===========================
 Input device: Default device @ Sample rate: 16000 Hz
 Playback commandline: aplay -Dhw:0,0 WAV_FILE

 ===========================================================
 ==         STARTING TO RECORD, MAKE SOME NOISE!          ==
 ===========================================================
 ===========================================================
 ==           DONE RECORDING, PLAYING BACK...             ==
 ===========================================================
Playing WAVE '/tmp/test.wav' : Signed 16 bit Little Endian, Rate 16000 Hz, Mono
aplay: set_params:1339: Sample format non available
Available formats:
- S32_LE
An error occured while playing back audio (1)
Restarting Mycroft voice service...
Already up to date.
Initializing...
Starting background service audio
Already up to date.
Initializing...
Starting background service voice
HinTak commented 3 years ago

@Drizzt321 I see you are doing aplay ...hw:0,0... - that's likely wrong... There is another phone jack on the respeaker board itself, and it is 1,0!

lxne commented 3 years ago

@HinTak I did send you some Euros, hope you can use 'em.

HinTak commented 3 years ago

Thanks @lxne, I appreciate that!