espressif / esp-idf

Espressif IoT Development Framework. Official development framework for Espressif SoCs.
Apache License 2.0
13.71k stars 7.3k forks source link

[TW#19688] I2S PDM Microphone: data is not correct (IDFGH-3491) #1758

Closed FlorisColpaert closed 6 years ago

FlorisColpaert commented 6 years ago

Hey, I have issue's using the INMP621 microphone with PDM.. I can't get correct data from it to calculate the noise level(dB). When there is no special sound. The data is always around 976 and doesn't gets to a negative value. I do se change when i make some noice, but i don't think it's relevant. The sound level only changes like 0.01db.

Does anyone see some major errors I made?

i2s_config_t i2s_config = { .mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX | I2S_MODE_PDM), .sample_rate = 44100, .bits_per_sample = I2S_BITS_PER_SAMPLE_16BIT, .channel_format = I2S_CHANNEL_FMT_ONLY_RIGHT, .communication_format = (i2s_comm_format_t)(I2S_COMM_FORMAT_PCM ), .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1, // lowest interrupt priority .dma_buf_count = 8, .dma_buf_len = 68, .use_apll = 0 };

// // Clock is internal // // No data out i2s_pin_config_t pinConfig = { .bck_io_num = I2S_PIN_NO_CHANGE, .ws_io_num = clkOut->GetPin(), // WS is connected to clock for external clock .data_out_num = I2S_PIN_NO_CHANGE, .data_in_num = pinRx->GetPin() // Data input of mic };

i2s_driver_install(I2S_NUM_0, &i2s_config, 0, NULL); //install and start i2s driver i2s_set_pin(I2S_NUM_0, &pinConfig);

.....

while ((readcount < 8000)) { if ( (i2s_pop_sample(I2S_NUM_0, (char *)&sample, (TickType_t)portMAXDELAY)) >= 2) { // sample = ((sampleBfr[1] << 8 & 0xFF00) + (sampleBfr[0] & 0x00ff)); rms += pow((double)sample/32767, 2); readcount++; } }

if( readcount >= 8000) { double db; if (rms != 0){ db = 20*log10(sqrt((double)rms/(readcount))); } else db = 0;

successHandler((float)db);

}else{ failHandler(); }

kewalmshah commented 6 years ago

Hi @FlorisColpaert can you please test the output data from microphone first ? I have a sample code tested and works fine with PDM microphones.

/* I2S PDM example */
#include <stdio.h>
#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "driver/i2s.h"
#include "esp_system.h"
#include "esp_spi_flash.h"
#include "rom/spi_flash.h"

#define SAMPLE_RATE    16000
#define BITS_PS        16
#define I2S_NUM        0
#define BUF_COUNT      4
#define BUF_LENGTH     250

/*
 * In PDM mode the clock is generated on WS line. Fwsclk = 64*(sample frequency)
 * Here sample frequuency is 8KHz, hence WS clk frequency will  be 64*8k = 512KHz
 * Each sector of flash is 4096 bytes
 */

int16_t samples_data[SPI_FLASH_SEC_SIZE / 2] = {0};   //Since each sample is 2 bytes

void app_main()
{
    i2s_config_t i2s_config = {
        .mode = I2S_MODE_MASTER | I2S_MODE_RX | I2S_MODE_PDM,    //master, RX, PDM
        .sample_rate = SAMPLE_RATE,
        .bits_per_sample = BITS_PS,                                    
        .channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT,
        .communication_format = I2S_COMM_FORMAT_I2S, //pcm data format
        .dma_buf_count = BUF_COUNT,                   
        .dma_buf_len = BUF_LENGTH,                  
        .use_apll = 0,                               //apll disabled
        .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1     //interrupt level 1(lowest priority)
    };

    i2s_pin_config_t pin_config = {
        .ws_io_num = GPIO_NUM_19,
        .data_in_num = GPIO_NUM_35,
    };

    i2s_driver_install(I2S_NUM, &i2s_config, 0, NULL);
    i2s_set_pin(I2S_NUM, &pin_config);
    printf("Start speaking\n");
    vTaskDelay(1000/portTICK_PERIOD_MS);
/*
 * Steps to follow:
 * 1. erase each sector
 * 2. read data from PDM mics
 * 3. write to flash
 */
    for (uint32_t base_addr = 0x300000; base_addr < 0x400000; base_addr += SPI_FLASH_SEC_SIZE) {

        printf("erasing sector %x\n", base_addr / SPI_FLASH_SEC_SIZE);
        spi_flash_erase_sector(base_addr / SPI_FLASH_SEC_SIZE);

        i2s_read_bytes(I2S_NUM, (const char*)samples_data, SPI_FLASH_SEC_SIZE, portMAX_DELAY);

        printf("writing at %x\n", base_addr);
        spi_flash_write(base_addr, samples_data, sizeof(samples_data));

        printf("done at %x\n", base_addr);
    }
    printf("Done writing data to flash\n");
}

Please follow these steps to test PDM mic output.

  1. Flash the attached test code and start speaking so that PDM mic can record your voice and store the sample data into flash.

  2. Note the base address for flash and size of the sample data(in our test code size is 0x100000 and start address is 0x300000).

  3. Run following commands: A. cd ~/esp-idf/components/esptool_py/esptool/ B. python esptool.py --chip esp32 --port /dev/ttyUSB3 --baud 1152000 read_flash 0x300000 0x100000 ~/test.pcm python esptool.py --chip esp32 --port [port name] --baud [baud rate] read_flash [start address] [size] [file name and location] Now you have stored the recorded mic data in file test.pcm

  4. Now play the recorded pcm data on your computer using following command: aplay -f S16_LE -c2 -r16000 test.pcm aplay -f [signed 16 little endian] -[stereo] -r[sample rate] [file name] Now you can listen to the recorded data.

FlorisColpaert commented 6 years ago

Hey, Sorry for my late response. The microphone had an offset error. I 'm sorry for taking your time. Now everything works fine now.

Piocky commented 6 years ago

Hi @FlorisColpaert, i'm also using the INMP621 with the ESP32, and i have some troubles to make it work properly. I tried the code given by @kewalmshah to record samples and then play them with Audacity, but when i'm increasing the sample rate (it should be more than 16kHz if i want to reach the 3.072MHz recommended by the datasheet), i get records with cuts in it and poor quality.

So i would like to know which parameters are you using to make them work properly, maybe your code if possible.

Thanks!

FlorisColpaert commented 6 years ago

The sample rate was set to 48000. And buffer count of 16. This is allowed if you sample only one channel. All other settings are still like in the source code above.

To read data I now use the function
result = i2s_read_bytes(I2S_NUM_0, (char*)array, 68, (TickType_t)portMAX_DELAY); if (result > 0) { Send stuff... }

But I use this only to calculate db's. Not to play audio. I sampled audio and send it to the pc. But keep in mind you have to send the data fast enough and have a large enough buffer. I had problems with small gaps in the data at the start. minimum data transfering with 48Khz*16bits is 768Khz bps. (bits)

Keep in mind the PDM to PCM only has to possible sample rates, the 4868 and 48128. Output is always at 48Khz and 16bits. (I2S_RX_PDM_SINC_DSR_16_EN). So the 48*68 should do the job. Setting other parameters could result in incorrect sampling.

Piocky commented 6 years ago

So you think i can't do the same with both channels connected to the same I2S port?

And so, maybe in my case the problem can be the speed for writing data on flash? That could be what cause the cuts in records?

FlorisColpaert commented 6 years ago

Never tried it. But just keep in mind if you setup stereo, you have to be able to write 48Khz*32b so 1.5Mbps. I don't know how fast you could write to the flash.. Then I think you should try to use a codec? But I don't have any experience with that. But cuts in audio are cost by to slow buffering or playing the data.

Piocky commented 6 years ago

Okay, i will try to investigate with that in mind. Thanks a lot for your help!

Piocky commented 6 years ago

I just found a simple way to do what i wanted to do:

I can now use a sample rate of 96kHz (48kHz as sample frequency), and playing the recorded sample without any cut.

faaltunel commented 4 years ago

Hi, I tried the above codes in order to get simple data from SPG08P4HM4H-1 (MEMS Digital Mic with Single Bit PDM Output) but I get all the same data. @kewalmshah I used your code with small save and print addition as shown below (just to print data on terminal):

`/ I2S PDM example /

include

include

include "freertos/FreeRTOS.h"

include "freertos/task.h"

include "driver/i2s.h"

include "esp_system.h"

include "esp_spi_flash.h"

include "rom/spi_flash.h"

include "esp_log.h"

define SAMPLE_RATE 16000

define BITS_PS 16

define I2S_NUM 0

define BUF_COUNT 4

define BUF_LENGTH 250

/*

int16_t samples_data[SPI_FLASH_SEC_SIZE / 2] = {0}; //Since each sample is 2 bytes

define I2S_size 8192

int32_t samples[I2S_size]; size_t bytes_read = 0;

static const char* TAG = "I2S:";

void app_main() { i2s_config_t i2s_config = { .mode = I2S_MODE_MASTER | I2S_MODE_RX | I2S_MODE_PDM, //master, RX, PDM .sample_rate = SAMPLE_RATE, .bits_per_sample = BITS_PS, .channel_format = I2S_CHANNEL_FMT_ONLY_RIGHT, .communication_format = (i2s_comm_format_t)(I2S_COMM_FORMAT_PCM ), //pcm data format .dma_buf_count = BUF_COUNT, .dma_buf_len = BUF_LENGTH, .use_apll = 0, //apll disabled .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1 //interrupt level 1(lowest priority) };

i2s_pin_config_t pin_config = {
    .ws_io_num = GPIO_NUM_19,
    .data_in_num = GPIO_NUM_35,
};

i2s_driver_install(I2S_NUM, &i2s_config, 0, NULL);
i2s_set_pin(I2S_NUM, &pin_config);
vTaskDelay(1000/portTICK_PERIOD_MS);

while(1)
{
    ESP_LOGW(TAG,"Start Talking in 3 secs:");
    vTaskDelay(3000/portTICK_RATE_MS);
    i2s_read(I2S_NUM_0,(char *)samples,8192,&bytes_read,portMAX_DELAY);
    for(int k=1000; k<2000; k++)
    {
        ESP_LOGW(TAG,"s[%d] = %d", k, samples[k]);
    }
    vTaskDelay(20000/portTICK_RATE_MS);
}

}`

But I only get below:

I (197) boot: Loaded app from partition at offset 0x10000 I (197) boot: Disabling RNG early entropy source... I (198) cpu_start: Pro cpu up. I (201) cpu_start: Application information: I (206) cpu_start: Project name: esp32-i2s-driver-example I (213) cpu_start: App version: 1 I (217) cpu_start: Compile time: Jun 11 2020 20:43:27 I (223) cpu_start: ELF file SHA256: 028c72052adb1165... I (229) cpu_start: ESP-IDF: v4.1-beta1-317-g84b51781c I (236) cpu_start: Starting app cpu, entry point is 0x40081044 I (0) cpu_start: App cpu up. I (246) heap_init: Initializing. RAM available for dynamic allocation: I (253) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM I (259) heap_init: At 3FFB49B8 len 0002B648 (173 KiB): DRAM I (265) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM I (272) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM I (278) heap_init: At 40089E58 len 000161A8 (88 KiB): IRAM I (284) cpu_start: Pro cpu start user code I (302) spi_flash: detected chip: generic I (303) spi_flash: flash io: dio W (303) spi_flash: Detected size(4096k) larger than the size in the binary image header(2048k). Using the size in the binary image header. I (313) cpu_start: Starting scheduler on PRO CPU. I (0) cpu_start: Starting scheduler on APP CPU. I (325) I2S: DMA Malloc info, datalen=blocksize=500, dma_buf_count=4 I (335) I2S: PLL_D2: Req RATE: 16000, real rate: 32258.000, BITS: 16, CLKM: 31, BCK: 5, MCLK: 31.250, SCLK: 1032256.000000, diva: 64, divb: 16 Start speaking W (1345) I2S:: Start Talking in 3 secs: W (4545) I2S:: s[1000] = -2027321559 W (4545) I2S:: s[1001] = -2027321559 W (4545) I2S:: s[1002] = -2027321559 W (4555) I2S:: s[1003] = -2027321559 W (4555) I2S:: s[1004] = -2027321559 W (4555) I2S:: s[1005] = -2027321559 W (4565) I2S:: s[1006] = -2027321559 W (4565) I2S:: s[1007] = -2027321559 W (4575) I2S:: s[1008] = -2027321559 W (4575) I2S:: s[1009] = -2027321559 W (4585) I2S:: s[1010] = -2027321559 W (4585) I2S:: s[1011] = -2027321559 W (4585) I2S:: s[1012] = -2027321559 W (4595) I2S:: s[1013] = -2027321559 W (4595) I2S:: s[1014] = -2027321559 W (4605) I2S:: s[1015] = -2027321559 W (4605) I2S:: s[1016] = -2027321559 W (4615) I2S:: s[1017] = -2027321559 W (4615) I2S:: s[1018] = -2027321559 W (4615) I2S:: s[1019] = -2027321559 W (4625) I2S:: s[1020] = -2027321559 W (4625) I2S:: s[1021] = -2027321559

Later I found that after 2000 samples it gives below data on terminal :

W (183265) I2S:: s[2042] = -2027321559 W (183265) I2S:: s[2043] = -2027321559 W (183275) I2S:: s[2044] = -2027321559 W (183275) I2S:: s[2045] = -2027321559 W (183285) I2S:: s[2046] = -2027321559 W (183285) I2S:: s[2047] = -2027321559 W (183295) I2S:: s[2048] = 0 W (183295) I2S:: s[2049] = 0 W (183295) I2S:: s[2050] = 0 W (183305) I2S:: s[2051] = 0 W (183305) I2S:: s[2052] = 0 W (183305) I2S:: s[2053] = 0 W (183315) I2S:: s[2054] = 0 W (183315) I2S:: s[2055] = 0 W (183325) I2S:: s[2056] = 0 W (183325) I2S:: s[2057] = 0 W (183325) I2S:: s[2058] = 0

Thanks in advance

phorenabess commented 3 years ago

Do you need a PDM to I2S converter, hardware component before you can convert PDM signal to I2S dB data? We want to output I2S db/SPL data using a microphone as a sensor rather than using it to record and play sound.

giPeteR commented 3 years ago

You don't need a converter. I've only tried ESP32 and its I2S interface has a built in PDM-converter. But pay attention to the Down-sample rate / Decimating filter. ESP32 can only use mic's with x64 or x128 oversampling rate. I unfortunately didn't understand this and bought mic's to detect Bats. But the mic's had x50 or x25 which doesn't work here. Well, they work but you can't use the extended frequency range up to 80kHz.

I'm using the Arduino platform (Platform-IO + VS Code) here due to some other features, although this would work in ESP-IDF as well. I prefer ESP-IDF if possible and I've done way more advanced things with ESP-IDF.

The mic I'm using is an InvenSense (TDK) ICS-41352 (PDM). I've also used a SPH0645LM4H (I2S).

My initiating code looks like this:

void setupMic(int mode) {
    esp_err_t err;
    i2s_config_t i2s_config;
    i2s_pin_config_t pin_config;

    if(mode == I2S_MODE_PDM) {
        Serial.println("Configuring PDM...");
        i2s_config.mode = i2s_mode_t(I2S_MODE_MASTER | I2S_MODE_RX | I2S_MODE_PDM);         // Receive, not transfer
        i2s_config.sample_rate = SAMPLE_FREQ;
        i2s_config.bits_per_sample = I2S_BITS_PER_SAMPLE_16BIT;             // PDM uses 16bit.
        i2s_config.channel_format = I2S_CHANNEL_FMT_ONLY_LEFT;
        i2s_config.communication_format = i2s_comm_format_t(I2S_COMM_FORMAT_I2S);
        i2s_config.intr_alloc_flags = ESP_INTR_FLAG_LEVEL1;                 // Interrupt level 1
        i2s_config.dma_buf_count = 4;                                       // number of buffers
        i2s_config.dma_buf_len = BLOCK_SIZE;                                // samples per buffer
        i2s_config.use_apll = true;

        pin_config.bck_io_num = -1;                                     // Not used in PDM.
        pin_config.ws_io_num = 33;                                      // I2S Clock out.
        pin_config.data_out_num = -1;                                   // I2S Out, Not used here so -1, i e no change.
        pin_config.data_in_num = 35;                                    // I2S Data in.

    } else {


sample_rate is the audio sample rate. The clock out will be x64 or x128 to the mic.
anhnhancao commented 3 years ago

Hey, Sorry for my late response. The microphone had an offset error. I 'm sorry for taking your time. Now everything works fine now.

Can i have more informations about the offset error of this microphone and how to remove it in code. Thank you very much