The Max78000 FTHR, while running the kws_20_demo with a 3.5mm microphone input, is experiencing issues

[ChongHao-Qiu]

I am working on developing an IOT device based on Max7800 FTHR. However, I faced an error when I tried to use the 3.5mm audio input with my own microphone on the FTHR board running kws20_demo. I followed the readme file in the “MAX78000/CNN/kws20_demo/” project and enabled ENABLE_CODEC_MIC in the project.mk file. But I got an error in the image below when executing the max9867_init(mc_i2c_regs_t *i2c_inst, int mclk, int controller) function.

There seems to be something wrong with the reg_read function in max9867.c and the control mode of the I2C1. Could you help me check this problem? I want to use my own microphone in the kws20_demo.

[Jake-Carter]

Hi @ChongHao-Qiu, thanks for reporting. We will re-test the project to see if we can replicate the issue

[Jake-Carter]

We were able to replicate the issue and are investigating the codec driver code

[Jake-Carter]

Sorry for the delay @ChongHao-Qiu,

The following diff fixes the codec drivers. It turns out the I2C was never initialized...

diff --git a/Libraries/MiscDrivers/CODEC/max9867.c b/Libraries/MiscDrivers/CODEC/max9867.c
index d895fa0c81..173704495a 100644
--- a/Libraries/MiscDrivers/CODEC/max9867.c
+++ b/Libraries/MiscDrivers/CODEC/max9867.c
@@ -377,6 +377,12 @@ int max9867_init(mxc_i2c_regs_t *i2c_inst, int mclk, int controller)
     if (!i2c_inst)
         return E_NULL_PTR;

+    if ((err = MXC_I2C_Init(i2c_inst, 1, MAX9867_ADDR)) != E_NO_ERROR)
+        return err;
+
+    if ((err = MXC_I2C_SetFrequency(i2c_inst, 100000)) != E_NO_ERROR)
+        return err;
+
     /* Static I2C request values */
     i2c_req.i2c = i2c_inst;
     i2c_req.addr = MAX9867_ADDR;

However there seems to be some deeper issues with the I2S drivers that we have been troubleshooting. It will take us some more time to test and fix them. I hope that the on-board microphone is acceptable as an alternative for you in the meantime.

[ChongHao-Qiu]

The on-board microphone is functioning correctly. However, having the 3.5mm audio input operational is essential for my project. Could you provide some guidance on rectifying the I2S drivers? I am attempting to rewrite and resolve this issue.

[Jake-Carter]

One of the main issues with our I2S drivers is that they don't seem to be well implemented for sample sizes other than 8, 16, or 32 bits. (see the logic in MXC_I2S_RevA_ConfigData) for example...

To sample the line-input, we need to interface with the on-board MAX9867 codec, with the codec acting as the master. The codec has 16 bits per sample (16-bit ADCs), and the default configuration from the drivers is a 24kHz sample rate (controlled by LRCLK).

The number of BCLKs per channel on the I2S bus is controlled by the BSEL register. The easiest options to use are the LRCLK multipliers.

• 64x LRCLK results in 32 bits per channel.
• 48x LRCLK results in 24 bits per channel. This is the default setting used by the drivers.

As a result, we have 24 bits per channel, but only 16 bits of actual data. These extra bits correspond to this section in the timing diagrams from the codec datasheet.

For the MAX78000, this configuration matches a sub-sample configuration with a sample size of 16 and a "bits per word" size of 24. See section 15.5.4 of the MAX78000 User Guide for more details on the register settings for this. The UG shows an 8-bit sub-sample as an example.

However, you'll notice that the conditional in our drivers here is not correct. It ends up setting a sample size of 16 and a "bits per word" size of 16, which results in a byte misalignment in the received data.

The following diff on the I2S_DMA_Target example's main file seems to work to trick the drivers into using uint32_t data alignment, sample size = 16, and "bits per word" = 24.

--- a/Examples/MAX78000/I2S_DMA_Target/main.c
+++ b/Examples/MAX78000/I2S_DMA_Target/main.c
@@ -180,7 +180,7 @@ void i2s_init(void)
 #define I2S_CRUFT_PTR (void *)UINT32_MAX
 #define I2S_CRUFT_LEN UINT32_MAX

-    req.wordSize = MXC_I2S_DATASIZE_HALFWORD;
+    req.wordSize = MXC_I2S_DATASIZE_WORD;
     req.sampleSize = MXC_I2S_SAMPLESIZE_SIXTEEN;
     req.justify = MXC_I2S_MSB_JUSTIFY;
     req.wsPolarity = MXC_I2S_POL_NORMAL;
@@ -200,6 +200,8 @@ void i2s_init(void)
     else
         printf("I2S initialized successfully \n");

+    // Overwrite bits-per-word = 24 after initialization
+    MXC_SETFIELD(MXC_I2S->ctrl1ch0, MXC_F_I2S_CTRL1CH0_BITS_WORD, (24 - 1) << MXC_F_I2S_CTRL1CH0_BITS_WORD_POS);

So bringing it together...

1. I would suggest starting with the I2S_DMA_Target example, and applying the workaround above. See if you can get the recording/playback on the line-in and line-out working with the codec alone. This would be a good starting point for digging into the driver's configuration and confirming a working setup.

2. From that point, the KWS model requires audio with a 16kHz sample rate and 16-bits per channel. You should be able to easily reduce the LRCLK setting in the codec to achieve the 16kHz sample rate.

3. From there, you can start moving towards the CNN again. The KWS demo doesn't use DMA to unload the I2S FIFOs, and I think part of the problem is that its I2S request struct (here) is not configured properly to match the codec if ENABLE_CODEC_MIC is defined. The demo program has a lot going on, so you might also find it valuable to look at the "Kown Answer Test" (KAT) in the kws20_v3 example. This is the direct output of the ai8x-synthesis tool for the model.

I hope this is helpful for getting started. We are also looking into this in parallel, but we are a relatively small team with lots of micros to maintain. Any contributions you're willing to make would be very much welcome, and we're happy to assist with more clarification however we can. Our I2S drivers have needed a refresh for some time.

[ahmetalincak]

Hello @ChongHao-Qiu ,

Please check if PR #847 fixes your issues. After configuring the example plug cable to J5 (line-in jack) and play a recorded sound to see KWS outputs. Please let us know if anything appears.

Note: Enabling 3.5mm Jack input doesn't allow you to use your headphone's analog mic as it's not connected to anywhere. Please see in schematics for the details.

[ChongHao-Qiu]

Hi, @ahmetalincak. I appreciate your help. I updated the latest SDK and built the kw_20 project successfully. However, I still get an error message in the serial port output: Have you happened to see this error?

[ahmetalincak]

Hi @ChongHao-Qiu,

Thanks for checking this. You can see in the PR that I added codec_init function to initialize MAX9867 and this would resolve any MAX9867 related problem.

As #847 has not been merged to main branch yet, I need to ask you to check if you checked out to branch _fix/i2s_driverenhancements (you can simply do it by running git fetch && git checkout -b fix/i2s_driver_enhancements origin/fix/i2s_driver_enhancements).

[Jake-Carter]

847 merged, closing as fixed