Audio filter work for CircuitPlayground Express

[tannewt]

Digital signal processing isn't my forte. Others have volunteered to come up with the best processing system for changing the PDM mic signal into PCM to be recorded to memory or SPI flash.

Right now the goal is 8bit, ~8khz just to get something going. If we can squeeze more out of the M0 then that'd be awesome.

Lets coordinate here what work people have done.

So, far I've been working on the pdm branch and copied code from this blog post. I've also posted some sample files here that were taken directly from the PDM lines using a Saleae. I believe the incoming data to the filter is correct.

[cyborg5]

I'm also interested in digital signal processing on the Circuit Playground Express. Not necessarily in Circuit Python. Regular Arduino C++ would be fine. My specific application is I need a bandpass filter to detect when someone is making a clicking sound with their tongue. I happen to know that clicks are about 11 kHz. This would be used for assistive technology for the disabled applications. So if anyone knows anything about how to create a digital bandpass filter that recognizes particular frequencies that would be very useful.

[papahabla]

After reviewing the code, I've identified some top level areas for improvement.

The first stage where a hamming weight is used to determine the value for each incoming PDM byte uses about 16 operations. This is done as an alternative to the original implementation which used a look-up table to save memory. Another alternative could be to use the lookup table but creating and initializing it on the stack/heap. Yet, another alternative could be to calculate the byte value directly instead of using the hamming window to count the number of ones as an intermediate.

DC offset is currently recalculated for each DMA buffer and based on a short time window - seems that DC should be removed using 1st order high-pass filter or other lightweight approach based on more history.

The final truncation to 8bit audio only shifts by 2bits where with a theoretical 14bit signal may need up to 5 bits and should be saturated to prevent integer wrapping issues. Optimal bit shift should be tested with hardware.

[ghost]

Unless I missed something, I think the only major issue here is that the filter doesn't do the full decimation. The Hamming weight does an initial decimation by 8 but you'd have to decimate again by some factor to reach the target sample rate.

[papahabla]

Currently, there are two stages of decimation. The initial 8:1 decimation used as input to the two integrator stages, and then the loop on self->bytes_per_sample = 16 completes the decimation to 1/128th the PDM sampling rate input to the comb sections.

[papahabla]

I'm checked out to the "pdm" branch and trying to test on hardware using the following which I plucked from PDMin.c comment:

import audiobusio
import board

# Prep a buffer to record into
b = bytearray(200)
with audiobusio.PDMIn(board.MICROPHONE_DATA, board.MICROPHONE_CLOCK) as mic:
    mic.record(b, len(b))

But I get the following error.

Traceback (most recent call last):
  File "main.py", line 7, in <module>
ValueError: Invalid clock pin

I tracked down the method raising the exception and hardcoded in what I thought are correct values and it seems to be working now.

void common_hal_audiobusio_pdmin_construct(audiobusio_pdmin_obj_t* self,
                                           const mcu_pin_obj_t* clock_pin,
                                           const mcu_pin_obj_t* data_pin,
                                           uint32_t frequency,
                                           uint8_t bit_depth,
                                           bool mono,
                                           uint8_t oversample) {
    self->clock_pin = &pin_PA10;
    self->clock_unit = 0;
    /* FIXME: 
    self->clock_pin = clock_pin; // PA10, PA20 -> SCK0, PB11 -> SCK1
    if (clock_pin == &pin_PA10 || clock_pin == &pin_PA20) {
        self->clock_unit = 0;
    } else if (clock_pin == &pin_PB11) {
        self->clock_unit = 1;
    } else {
        mp_raise_ValueError("Invalid clock pin");
    }
    */

    self->data_pin = &pin_PA08;
    self->serializer = 1;

    /* FIXME: 
    self->data_pin = data_pin; // PA07, PA19 -> SD0, PA08, PB16 -> SD1
    if (data_pin == &pin_PA07 || data_pin == &pin_PA19) {
        self->serializer = 0;
    } else if (data_pin == &pin_PA08
        #ifdef PB16
        || data_pin == &pin_PB16) {
        #else
        ) {
        #endif
        self->serializer = 1;
    } else {
        mp_raise_ValueError("Invalid data pin");
    }
    */

[tannewt]

Awesome! I think your arguments are backwards, clock pin first and then data.

Here is my test code:

import board
import audioio
import audiobusio
import digitalio
import time
import array
import math

buf = bytearray(8000)

print(3)
time.sleep(1)
print(2)
time.sleep(1)
print(1)
time.sleep(1)
print("recording", time.monotonic())
trigger = digitalio.DigitalInOut(board.A1)
trigger.switch_to_output(value = True)
with audiobusio.PDMIn(board.MICROPHONE_CLOCK, board.MICROPHONE_DATA) as mic:
    mic.record(buf, len(buf))
trigger.value = False
print("done recording", time.monotonic())

for i in buf[:10]:
    print(i)
print(min(buf), max(buf))
time.sleep(0.1)

speaker_enable = digitalio.DigitalInOut(board.SPEAKER_ENABLE)
speaker_enable.switch_to_output(value = True)
trigger.value = True

length = 16000 // 440
b = array.array("H", [0] * length)
for i in range(length):
    b[i] = int(math.sin(math.pi * 2 * i / length) * (2 ** 15 - 1) + (2 ** 15))

print(b)
with audioio.AudioOut(board.SPEAKER, b) as sample:
    sample.play(loop=True)
    time.sleep(1)
    sample.stop()

trigger.value = False

time.sleep(1)

trigger.value = True
print("playback", time.monotonic())
with audioio.AudioOut(board.SPEAKER, buf) as speaker:
    speaker.frequency = 8000
    speaker.play()
    while speaker.playing:
        pass

trigger.value = False

[papahabla]

Yes, thanks, backwards indeed! I guess I'm still adjusting to not having a debugger.

I'm almost ready for a PR for branch but still have some more testing to do. Not a lot changed actually.

1. I used a high pass filter for DC removal, which is implemented as a fixed-point single pole filter.

2. The final shiftout is set to 2bits based on the level of the example files, leaving some room for closer to the microphone or louder speech situations.

3. Also saturation to 8bit audio signal is performed which should help for when the 8bit signal is overdriven a bit. You'll still get some distortion but it will be more desirable than overflow.

I think it's going to be hard to get good speech intelligibility at 8kHz. The CIC filtering has a side effect of low passing the signal and that seems to significantly attenuate <4kHz. The high rate PDM example file sounds pretty good but it's ~22kHz sampling rate. Perhaps compromising at something around 16kHz would work well. Another approach I've seen mentioned in papers is adding a mitigating filter but that's not free and can add processing MCPS.

The plot below compares the frequency response of the CIC filter at 8kHz vs 16kHz audio sampling rate.

I'm not familiar with configuring DMA and wondering what needs to be done to adjust the PDM rate to ~2.048mhz. I'll look into some kind of low MCPS filter, too.

Also, it would be helpful to write to file from the REPL but the following seems to return error 10. This is probably a bad question but is there any way to enable this? Then I can analyze the recorded signal using desktop python/audio tools.

b = bytearray(2000)
with open('/flash/test.pcm', 'wb') as f:
    f.write(b)

[tannewt]

The DMA is clocked by the I2S peripheral and its clock is configured here. (Currently GCLK3 is 8mhz and is divided by 7.) The frequency that the filter runs is impacted by the number of samples per buffer and the number of bytes per sample. Changing those will be a trade-off of memory vs timing resilience.

Yeah, right now the flash isn't writeable because its writeable by USB. I intend on adding a way to change this on bootup but haven't yet. You can try returning zero here but beware that it may lead to filesystem corruption if your host os is writing at the same time you are.

[papahabla]

Thanks, I can now write files! It's a bit tricky because the desktop os doesn't get updated after files are written so I need to unplug/replug the CP Express to open them, but it's still super helpful.

[tannewt]

FYI, @ladyada set me up with a sinc filter which was submitted in 557ceded00de20d2070e1397d3dec322f753d1e6.

[papahabla]

@tannewt, thanks for the heads-up. Looks interesting, I'll take it for a test drive.

[tannewt]

I think the audio filtering is ok for now. We can create a new issue if/when it needs more love.

[kattni]

For anyone who is using the test code in this issue, be aware that we identified a necessary change. The frequency must be at least 16000 to work with the mic. Overall frequency must be at least 1MHz, and it is determined by multiplying oversample and frequency. The default for oversample is 64. It is possible this issue will be resolved in the future, but as it is, this change may fix issues you're having with the mic appearing not to be functional.