[Bug]: Static on Left Channel on Two Pi Zero (2-Mic Pi HAT)

[fanuch]

Describe the bug

Steps to reproduce the behavior:

1. Source two Raspi Zero
2. Install hats
3. Follow "first-steps" to install and record audio
4. Audio playback has static

Expected behavior

No static or scratching on playback

Platform

• Device: Pi Zero
• OS: Raspbian - Linux pi 5.10.52+ #1441 Tue Aug 3 18:08:18 BST 2021 armv6l GNU/Linux
• Version/commit number: b595b95b2184f52e752256cca17e9b92e1a19cd7 (latest on master at time of post)

Relevant log output

Image showing the recording from two new 2-Mic HATS and two separate Pi Zero on Audacity https://imgur.com/a/ZjWsmDY

Note that both left channels have the static, but the right channel is clear and has no static

[AIWintermuteAI]

Hi @fanuch !

I wasn't able to reproduce the bug you described with the following setup: Raspberry Pi Zero W v1.1 5.10.52+ kernel b595b95 reSpeaker 2-mics Pi Hat v1.0

Experiment: Record audio with arecord -f cd -Dhw:0 test.wav, then play it with arecord -f cd -Dhw:0 test.wav. Transfer it to PC for visualizing in Audacity.

Results: Clear sound. https://drive.google.com/file/d/1F1eAowiQwfwHFkSampaDt0uBd5VePur-/view?usp=sharing I was able the other issue (https://github.com/respeaker/seeed-voicecard/issues/282) is indeed present on Raspberry Pi Zero W, but no static you described.

Suggestions: I saw something very similar to bug you describe when I was testing the DTO and inbuilt Kernel driver for wm8960, described here https://github.com/respeaker/seeed-voicecard/issues/281. I don't have sound samples left, but overall result was same as I can see in your recording, noise in one channel. See this issue https://github.com/raspberrypi/linux/issues/4384 Make sure you are indeed using seeed-voicecard driver. Try testing with a)clean install of Raspberry Pi OS b) our image, which you can find in README.

[fanuch]

Thanks for the suggestions!

Try testing with a)clean install of Raspberry Pi OS

• I used Raspbian OS Lite burned with Etcher
• I applied the ssh and wpa_supplicant.conf files to the boot partition for access
• sudo apt update && sudo apt upgrade -y
• git clone https://github.com/respeaker/seeed-voicecard.git; cd seeed-voicecard; sudo ./install.sh
• sudo reboot
• sudo arecord -f cd -Dhw:0 test.wav

Still the same issue

This time I also get the https://github.com/respeaker/seeed-voicecard/issues/282 issue

I have flashed my second Pi Zero with the image from the README, but sudo arecord -l shows nothing. I'm confused as I though that image was already meant to have the relevant drivers?

Will persevere with that image

[DeeBeeKay]

For what it is worth, I am having this issue on a Raspberry Pi 3B+ that I just set up this morning, with the intention of running Rhasspy.

This is a brand new install of Buster, updated to the latest everything. The problem persists on both Ethernet and WiFi - I tested both in case the network was causing interference.

I had another brand new Respeaker 2-Mic on the shelf - I had bought two with the intention of having multiple nodes in the house. So I tried swapping the apparently problematic Respeaker 2-Mic with a new one. The same problem persists, even with fresh hardware.

The thing is basically unusable.

[fanuch]

@DeeBeeKay

For what it is worth, I am having this issue

You are seeing literally the same issue I am? Clicks on the left channel?

running Rhasspy. ... with the intention of having multiple nodes in the house

Same thing with me. I was hoping this was going to be as close to an out-of-the-box solution as a maker can have.

[DeeBeeKay]

You are seeing literally the same issue I am? Clicks on the left channel?

I have a single mono speaker plugged into the speaker output of the Respeaker Mic2. The audio is full of crackles and noise, and is unusable. I don't have your ability to map the stereo channels to show where the noise is.

[fanuch]

I don't have your ability to map the stereo channels to show where the noise is. Just run:

sudo arecord -f cd -Dhw:0 test.wav    # to record

Then on your local machine i.e. Windows or Mac, run:

scp pi@ip_address_here:~/test.wav . # to download

[fanuch]

Sorry, tried to cancel a message and clicked "close with comment" instead. Issue is still on-going

[AIWintermuteAI]

@fanuch @DeeBeeKay Interesting. Can you also specify the hardware revisions of reSpeaker and Raspberry Pi Zero W? To make sure we are testing the same hardware

[fanuch]

ReSpeaker 2-Mics Pi HAT v1.1 (for both)

Raspberry Pi Zero W v1.1 (for both)

[AIWintermuteAI]

Okay, my reSpeaker is rev 1.0 here, I need to get rev 1.1 for further testing then. Your Raspberry Pi is the same version as mine (1.1.), there are only two subtle differences: ANATEL number 40-pin header seems to be soldered manually (mine is pre-soldered)

[DeeBeeKay]

I managed to transfer the test.wav to my PC and opened it in Audacity, and was thus able to confirm that the crackles and noise are in the left hand channel.

In terms of hardware, the Raspberry Pi I am using doesn't seem to have a version number, it just says Raspberry Pi 3 Model B+.

Both of the Respeaker 2-Mic Pi HATs I have are version 1.0.

[fanuch]

@DeeBeeKay did you need to solder anything on to your ReSpeakers or your Pi? As far as I have seen, the Pi 3B has the headers soldered on from the factory - so too for the 2-Mic HAT.

[DeeBeeKay]

@fanuch No soldering was necessary, the headers were done at the factory.

Are you implying there could be solder somewhere there shouldn't? I have one other Pi3B+ I could test with, if you like, also with factory headers. Then, in a drawer somewhere, I have a Pi Zero W, but I'd need to solder a header onto that and my soldering skills are ... well, I guess they're functional.

[fanuch]

Are you implying there could be solder somewhere there shouldn't?

I'm trying to removing the chance that my header soldering may have caused the issue! That you didn't need to solder helps rule that out in my two Pi and two HATs :)

[DeeBeeKay]

I just did another test with a different rpi3B+. The noise is present in the left channel.

Here's why I think this is software related. I used this rpi and this Respeaker 2mic before to build a Picroft, about a year back. At that time, it worked without issue. The hardware is exactly the same, the only thing that has changed is the software.

[HinTak]

See the log message on https://github.com/respeaker/seeed-voicecard/commit/19067f3333d7759bdf626e62bc09a7deae4627a1 - the trigger start/stop order has changed in kernel 5.5 regarding the cpu end and the codec. This broke the 6-mics and linear-4-mics completely, but the 2-mics and square 4-mics was not (noticeably) affected. Maybe the 2-mics device driver needs to adjust for the 5.5+ kernel change too.

[HinTak]

Just to eliminate that possibility, see if downgrading the kernel to 5.4 helps. Instructions at https://github.com/HinTak/RaspberryPi-Dev/blob/master/Downgrading-Pi-Kernel.md

Disclaimer: just another user, I don't work for Seeed Studio...

[AIWintermuteAI]

We have discussed it internally and concluded it is likely NOT due to the difference between 1.0 and 1.1. I had a good look at the kernel version you mentioned @fanuch - I was a bit confused why you have 5.10.52 when latest official version downloaded from Raspberry Pi OS website is 5.10.17. It turns out during the driver installation kernel is upgraded to match the latest headers. Very recently WM8960 codec kernel module was added to kernel and now present by default. I think this is what causing the issue you're having. https://github.com/raspberrypi/linux/commit/f800047be45a5a811b1a550449647879c7d92e20#diff-120dd27ba4fda8b79a76fd6519cc804ca4b0877045961e75243355cedcae38a2 It was enabled in this commit.

During the installation process the default wm8960.ko file should be replaced with seeed studio version though. It seems that it didn't.

We added manual removal of default WM8960 kernel driver in the latest commit - but there is another issue in the script, which we are working on eliminating currently. You can try pulling the newest commit and running install.sh on top of already installed drivers (basically install twice).

Oh and by the way, unless necessary don't do sudo arecord or sudo aplay - these commands should work without sudo.

[AIWintermuteAI]

@HinTak Hmmm, probably not that issue. It is more likely to do with default WM8960 kernel driver being enabled in 5.10.52

[HinTak]

@AIWintermuteAI I just left a comment on that commit - the Linux kernel allows for multiple kernel modules of the same name, and dkms-managed modules overrides those from the rest.

Argh, I see there is a bug - you should not hard-code the module location. dkms's default location overrides the rest of the kernel, but not when you use a non-default location.

[HinTak]

Removing the three module location lines https://github.com/respeaker/seeed-voicecard/blob/17e2875d4d6b47376861a1f936b9f5b365845a11/dkms.conf#L6 might be a good idea.

[AIWintermuteAI]

I've narrowed it down to this line https://github.com/respeaker/seeed-voicecard/blob/17e2875d4d6b47376861a1f936b9f5b365845a11/install.sh#L130 Basically what happens is that when there is a new kernel present, it is downloaded and installed --- but before reboot it is not loaded. So uname -r command returns old kernel version, which does not exist anymore. That causes this for loop https://github.com/respeaker/seeed-voicecard/blob/17e2875d4d6b47376861a1f936b9f5b365845a11/install.sh#L151 to fail and no kernel modules are built.

It is not surprising, since this commit https://github.com/respeaker/seeed-voicecard/commit/60e90d6faec16309a619e060049feed9890e8490 mentions

and ask the user to reboot before continue

but the reboot prompt was never added.

[AIWintermuteAI]

The solution here is to check if kernel version has changed and if it was, ask user to reboot before continuing the installation.

[DeeBeeKay]

Hi @AIWintermuteAI. Following your advice, I pulled the latest version and installed it again. The problem is gone!

Thank you so much for looking into this.

[AIWintermuteAI]

@DeeBeeKay no problem! What actually fixed it was probably running the installation script again after reboot. @fanuch you can try doing that too. I'll make necessary changes, test and push them tomorrow.

[fanuch]

Gah beat me to it @DeeBeeKay!

I also pulled the latest commit and did a "dirty install" on latest commit https://github.com/respeaker/seeed-voicecard/commit/17e2875d4d6b47376861a1f936b9f5b365845a11.

Sound is clear and audible (maybe too clear, with that quiet ramping up on the right channel on issue https://github.com/respeaker/seeed-voicecard/issues/282 :) )

Thanks for your fast help @AIWintermuteAI. I'm really excited to continue my experimentation with the 2-Mic Pi HAT!

Thank you for your input too @HinTak; I'm certain a rollback would have resolved the issue - just wanted to test the removing of the default WM8960 kernel module first.

[fanuch]

Closing this issue as it looks to be resolved and debugged

[AIWintermuteAI]

Yes, it is resolved. Now if active kernel version doesn't match updated kernel version this message will be displayed Then after reboot when user runs the ./install.sh script again installation will proceed to building kernel module and replacing it (in case with WM8960.