IDF4.4 I2S PDM issue in using PDM output microphone, clk freq cannot up to 1MHz? (IDFGH-9771)

edword01 commented 1 year ago

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report

IDF version: v4.4

PDM microphone document

https://item.szlcsc.com/510337.html

ESP module:

ESP32S3-1u-N8R8

Hardware connection:

PDM output microphone with L/R pin, CLK pin and Data pin. ISSUE: my microphone requires clk up to 1MHz, if clk freq less than 1MHz, it will sleep. how to config , to make esp32s3's i2s interface can generate 1Mhz????

i2s config is showed bellow, I have try several days to config this, but all fail to do this. I config microphone's CLK pin to the 'pin_config.ws_io_num'.

i2s config:

/------------------------------------/ // Set the I2S configuration as PDM and 16bits per sample i2s_config_t i2s_config = { .mode = I2S_MODE_MASTER | I2S_MODE_RX | I2S_MODE_PDM, .sample_rate = I2S_SAMPLE_RATE, //CONFIG_EXAMPLE_SAMPLE_RATE, .bits_per_sample = I2S_BITS_PER_SAMPLE_32BIT, .channel_format = I2S_CHANNEL_FMT_ONLY_LEFT,//I2S_CHANNEL_FMT_ONLY_RIGHT,//I2S_CHANNEL_FMT_ONLY_LEFT, .communication_format = I2S_COMM_FORMAT_STAND_PCM_SHORT, .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1, //ESP_INTR_FLAG_LEVEL2, .dma_buf_count = 8, .dma_buf_len = 128, .use_apll = 0,//0 .fixed_mclk = 1000000, };

// Set the pinout configuration (set using menuconfig)
i2s_pin_config_t pin_config = {
    .mck_io_num = I2S_PIN_NO_CHANGE, //I2S_PIN_NO_CHANGE,
    .bck_io_num = I2S_PIN_NO_CHANGE, //MIC_SCK_PIN,
    .ws_io_num = MIC_SCK_PIN,//MIC_LR_PIN, //CONFIG_EXAMPLE_I2S_CLK_GPIO,
    .data_out_num = I2S_PIN_NO_CHANGE,
    .data_in_num = MIC_DAT_PIN, //CONFIG_EXAMPLE_I2S_DATA_GPIO,

};

// Call driver installation function before any I2S R/W operation.
ESP_ERROR_CHECK( i2s_driver_install(CONFIG_I2S_CH, &i2s_config, 0, NULL) );
ESP_ERROR_CHECK( i2s_set_pin(CONFIG_I2S_CH, &pin_config) );

/---------------------------------------------------------------------/

edword01 commented 1 year ago

this microphone status flow:

L-KAYA commented 1 year ago

Hi @edword01, I've tested the PDM clock output with your configuration on ESP32S3@IDF release/v4.4, it can reach about 3MHz.

May I confirm that how you monitor the clock output? Is it monitored on MIC_SCK_PIN with a logic analyzer or an oscilloscope?

BTW, although your configuration can output the clock, there are still some inappropriate config:

Please set bits_per_sample to I2S_BITS_PER_SAMPLE_16BIT, because PDM can only support I2S_BITS_PER_SAMPLE_16BIT;
fixed_mclk is not the output clock of PDM mode, it is at least 4 times than the signal on ws_io_num (i.e. the PDM clock output pin). And, fixed_mclk won't take effect because it relies on APLL while ESP32S3 doesn't support APLL.

edword01 commented 1 year ago

@L-KAYA thanks, now i can detected 1.4MHz clk, is it calculate PDM clock automatically??

how to configure PDM clock freq? if i want to.
now , PDM data signal out , only has about 40mv. very low.

edword01 commented 1 year ago

According to this : https://github.com/espressif/esp-idf/issues/8660#issuecomment-1205737159 can i need to change wav file's sample rate to 1/2 of the value set to the i2s config ?

edword01 commented 1 year ago

now i config :

// Set the I2S configuration as PDM and 16bits per sample
    i2s_config_t i2s_config = {
        .mode = I2S_MODE_MASTER | I2S_MODE_RX | I2S_MODE_PDM,
        .sample_rate = I2S_SAMPLE_RATE, //CONFIG_EXAMPLE_SAMPLE_RATE,
        .bits_per_sample = I2S_BITS_PER_SAMPLE_16BIT,
        .channel_format = I2S_CHANNEL_FMT_ONLY_RIGHT,//I2S_CHANNEL_FMT_ONLY_RIGHT,//I2S_CHANNEL_FMT_ONLY_LEFT,
        .communication_format = I2S_COMM_FORMAT_STAND_MSB,//I2S_COMM_FORMAT_STAND_PCM_SHORT,
        .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1, //ESP_INTR_FLAG_LEVEL2,
        .dma_buf_count = 8,
        .dma_buf_len = 128,
        .use_apll = 0,//0
        //.fixed_mclk = 1000000,
    };
    i2s_pcm_cfg_t pcm_config = {
      .pcm_type = I2S_PCM_A_COMPRESS,
    };

    //uint32_t pdm_clk = i2s_calculate_pdm_rx_clock(i2s_config.sample_rate, I2S_BITS_PER_SAMPLE_16BIT, I2S_CHANNEL_MONO, 1);
    //i2s_config.fixed_mclk = pdm_clk;
    // Set the pinout configuration (set using menuconfig)
    i2s_pin_config_t pin_config = {
        .mck_io_num = I2S_PIN_NO_CHANGE,//I2S_PIN_NO_CHANGE,
        .bck_io_num = I2S_PIN_NO_CHANGE,//MIC_LR_PIN, //MIC_SCK_PIN,
        .ws_io_num = MIC_SCK_PIN,//MIC_LR_PIN, //CONFIG_EXAMPLE_I2S_CLK_GPIO,
        .data_out_num = I2S_PIN_NO_CHANGE,
        .data_in_num = MIC_WS_DAT, //CONFIG_EXAMPLE_I2S_DATA_GPIO,

    };

    // Call driver installation function before any I2S R/W operation.
    ESP_ERROR_CHECK( i2s_driver_install(CONFIG_I2S_CH, &i2s_config, 0, NULL) );
    i2s_pcm_config(CONFIG_I2S_CH, &pcm_config);
    i2s_set_pdm_rx_down_sample(CONFIG_I2S_CH, I2S_PDM_DSR_16S);
    ESP_ERROR_CHECK( i2s_set_pin(CONFIG_I2S_CH, &pin_config) );

Then after save the wav file, the wav file wave look like this:

Is that communication_format config error?? the wave are not right when recording.

edword01 commented 1 year ago

@L-KAYA What's the config logic of communication format??? in PDM mode? about I2S_COMM_FORMAT_STAND_MSB or I2S_COMM_FORMAT_STAND_I2S, and I2S_CHANNEL_FMT_ONLY_RIGHT or I2S_CHANNEL_FMT_ONLY_LEFT. ? how to select this value in PDM mode, and is may affect the logic of esp32s3 to translate PDM to PCM file.

L-KAYA commented 1 year ago

In pdm mode, communication format takes no effect. The channel format can help you to select left / right or both PDM microphones.

Normally, i2s_pcm_config and i2s_set_pdm_rx_down_sample is not needed.

The output clock is influenced by the down_sample_rate

edword01 commented 1 year ago

but now the wav file is not right，looks only have noises.

---Original--- From: @.> Date: Tue, Apr 4, 2023 19:39 PM To: @.>; Cc: @.**@.>; Subject: Re: [espressif/esp-idf] IDF4.4 I2S PDM issue in using PDM outputmicrophone, clk freq cannot up to 1MHz? (IDFGH-9771) (Issue #11105)

In pdm mode, communication format takes no effect. The channel format can help you to select left / right or both PDM microphones.

Normally, i2s_pcm_config and i2s_set_pdm_rx_down_sample is not needed.

The output clock is influenced by the down_sample_rate

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

edword01 commented 1 year ago

@L-KAYA a little confusion here, what's the data format after using PDM mode to read data? data read from 'i2s_read' is PCM data ? and can it just combine wave header and save as wav file? or i need to do some conversion to the raw data ??

L-KAYA commented 1 year ago

Yes, the date read from I2S will be transmitted to PCM format by the hardware, no need to convert it manually.

If possible, I suggest to use PDM in IDF v5.0. Limited by the legacy APIs, the PDM supporting on v4.4 is not that completed.

edword01 commented 1 year ago

now i cannot obtain the wav file correctly，wave looks only have noise，don't know what to do now

---Original--- From: @.> Date: Thu, Apr 6, 2023 11:49 AM To: @.>; Cc: @.**@.>; Subject: Re: [espressif/esp-idf] IDF4.4 I2S PDM issue in using PDM outputmicrophone, clk freq cannot up to 1MHz? (IDFGH-9771) (Issue #11105)

Yes, the date read from I2S will be transmitted to PCM format by the hardware, no need to convert it manually.

If possible, I suggest to use PDM in IDF v5.0. Limited by the legacy APIs, the PDM supporting on v4.4 is not that completed.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

L-KAYA commented 1 year ago

now i cannot obtain the wav file correctly，wave looks only have noise，don't know what to do now

You can check the issue in following step:

Monitor the signals on the data line, as a PDM microphone, it should output the data as long as the clock is provided appropriately.

If there is data on line, please check if the data line is connected to the correct GPIO
If there is no data on line, keep going to the next check

Provide square wave at about several MHz to the PDM Mic directly by a signal generator(you can also generate the clock by other way), so that the PDM Mic can get a stable and appropriate clock.

If the data can be detected on line in this case, it means the I2S PDM not output an appropriate clock, we need to look into it then;
If the data still doesn't appear, the PDM Mic might be broken;

edword01 commented 1 year ago

@L-KAYA thanks, now i can record wav successfuly, but volume very low, how to do with this: And why the signal's horizontal axis not start from Inf ??

L-KAYA commented 1 year ago

As far as I know, I2S PDM doesn't support adjust the gain, the url of the Mic doc seems invalid so I don't know if the PDM Mic supports to increase the gain.

Not sure why the wave has offset, maybe you can try another software: audacity

xG3nesis commented 4 months ago

Hello guys @L-KAYA, @edword01 !

It seems like it's been a while since this issue was initially raised! I'm chiming in now with my comment, hoping to receive some assistance and possibly provide help to others who might come across this thread. Let's dive straight into it! 👇

I've bought an M5 Atom Echo on which there is an ESP32-PICO-D4 with an SPM1423 PDM microphone and a NS4168 speaker. Here is the schematic available on the website of M5 stack :

My goal was to record my voice using a microphone, transmit it via Bluetooth to my computer. However, when I attempted to implement this using PlatformIO with the Arduino ESP32 library version 2.0.15 (the latest), which is based on ESP-IDF v4.4.7, I encountered an issue. Despite setting the parameters to signed 16-bit PCM, MSB, Stereo, and a sample rate of 16000Hz (although it doesn't work when I set it to 44100Hz in my code), I couldn't recover my PCM audio when decoding it with Audacity. Here's what happens when importing the raw file:

Rather than PCM, it appears that I'm getting PDM raw data. It seems that automatic conversion from PDM to PCM is not occurring.

Here's a glimpse of my code:

#include <BLEDevice.h>
#include <BLEServer.h>
#include <BLE2902.h>
#include <driver/i2s.h>
#include <M5Atom.h>

#define MICRO_I2S_NUMBER I2S_NUM_0

#define CONFIG_I2S_BCK_PIN 19
#define CONFIG_I2S_LRCK_PIN 33
#define CONFIG_I2S_DATA_PIN 22
#define CONFIG_I2S_DATA_IN_PIN 23

#define MODE_MIC 0
#define MODE_SPK 1
#define DATA_SIZE 1024

uint8_t microphonedata0[1024 * 80];
int data_offset = 0;
int count = 0;

/*INITIALIZATION OF I2S PARAMETERS (MICROPHONE)*/
void InitI2SMic(int mode)
{
    // Configurate general I2S Driver parameters !
    esp_err_t err = ESP_OK;

    i2s_driver_uninstall(MICRO_I2S_NUMBER);
    i2s_config_t i2s_config = {
        .mode = (i2s_mode_t)I2S_MODE_MASTER,
        .sample_rate = 44100,
        .bits_per_sample = I2S_BITS_PER_SAMPLE_16BIT, 
        .channel_format = I2S_CHANNEL_FMT_ALL_RIGHT,
        .communication_format = I2S_COMM_FORMAT_STAND_I2S,
        .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,
        .dma_buf_count = 6,
        .dma_buf_len = 60,
        .use_apll = false,
        .tx_desc_auto_clear = false,
        .fixed_mclk = 0, 
    };

    if (mode == MODE_MIC)
    {
        i2s_config.mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX | I2S_MODE_PDM);
    }
    else
    {
        i2s_config.mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_TX);
        i2s_config.use_apll = false;
        i2s_config.tx_desc_auto_clear = true;
    }

    err += i2s_driver_install(MICRO_I2S_NUMBER, &i2s_config, 0, NULL);

    //Configurate I2S Driver pinout !
    i2s_pin_config_t tx_pin_config;

    tx_pin_config.mck_io_num = GPIO_NUM_0;
    tx_pin_config.bck_io_num = CONFIG_I2S_BCK_PIN;
    tx_pin_config.ws_io_num = CONFIG_I2S_LRCK_PIN;
    tx_pin_config.data_out_num = CONFIG_I2S_DATA_PIN;
    tx_pin_config.data_in_num = CONFIG_I2S_DATA_IN_PIN;

    //Serial.println("Init i2s_set_pin");
    err += i2s_set_pin(MICRO_I2S_NUMBER, &tx_pin_config);
    //Serial.println("Init i2s_set_clk");
    err += i2s_set_clk(MICRO_I2S_NUMBER, 44100, I2S_BITS_PER_SAMPLE_16BIT, I2S_CHANNEL_MONO);
}

/*SETTING UP BLUETOOTH LOW ENERGY PARAMETERS AND OTHER FUNCTIONS
...
*/

void loop() 
  {
    if (M5.Btn.isPressed())
    {
      InitI2SMic(MODE_MIC);
      M5.dis.drawpix(0, CRGB(128, 0, 0));
      output_token = true;

      size_t byte_read = 0;
      data_offset = 0;
      count = 0;

      Serial.println("Recording started !");

      while (data_offset < 81920)
      {
          i2s_read(MICRO_I2S_NUMBER, (char *)(microphonedata0 + data_offset), DATA_SIZE, &byte_read, (100 / portTICK_RATE_MS));
          data_offset += 1024;
          M5.update();
          count = count + byte_read;
          if (M5.Btn.isReleased() || data_offset >= 81919)
              break;
      }

      Serial.println("Recording Ended !");
    }

    if (deviceConnected) {
      M5.dis.drawpix(0, CRGB(0, 0, 128));
      M5.update();

      if (output_token == true) {

        Serial.println("Sending data to device !");

        int l_range = 0;

        Serial.println("");
        Serial.print("Taille de l'enregistrement vocal (Byte) : ");
        Serial.print(count);
        Serial.println("");
        Serial.print("Nombre de paquets BLE à transmettre (Taille/512) : ");
        Serial.print(floor(count/512));
        Serial.println("");

        for(size_t j = 0; j < floor(count/512); j = j + 1) {
          uint8_t sub[512];
          Serial.println("");
          Serial.print("Paquet n°");
          Serial.print(j);
          Serial.println("");
          pickSubarray(microphonedata0, sub, l_range, 512 * (j + 1));
          pTxCharacteristic->setValue(sub, 512);
          pTxCharacteristic->notify();
          l_range += 512;
          Serial.print("------------------------------");
          Serial.print("");
        };

        if(count%512 != 0) {
          uint8_t sub[count - l_range];
          pickSubarray(microphonedata0, sub, l_range, count);
          pTxCharacteristic->setValue(sub, count - l_range);
          pTxCharacteristic->notify();
        }

        output_token = false;
      };
    } else {
      if (output_token == true) {
        InitI2SMic(MODE_SPK);
        size_t bytes_written;
        i2s_write(MICRO_I2S_NUMBER, microphonedata0, data_offset, &bytes_written, portMAX_DELAY);
        output_token = false;
      }
    }

    M5.dis.drawpix(0, CRGB(0, 128, 0));
    delay(500);
    M5.update();

  }

It seems like you're suggesting that PDM data should be automatically converted to PCM, @L-KAYA.

Yes, the date read from I2S will be transmitted to PCM format by the hardware, no need to convert it manually.

If possible, I suggest to use PDM in IDF v5.0. Limited by the legacy APIs, the PDM supporting on v4.4 is not that completed.

Additionally, I experimented with the new version of Arduino (pre-release - version 3.0.0) built on ESP-IDF 5.1.2 in the Arduino IDE, using the following code:

#include "driver/i2s_pdm.h"
#include "driver/gpio.h"

#define CONFIG_I2S_CLK_PIN GPIO_NUM_33
#define CONFIG_I2S_DATA_IN_PIN GPIO_NUM_23
#define BUTTON_PIN GPIO_NUM_39

#define SAMPLE_RATE 44100
#define SAMPLE_SIZE (16 * 1024)

i2s_chan_handle_t rx_handle = NULL;

uint16_t microphone_data[SAMPLE_SIZE];
size_t bytes_read;

void init_microphone(void)
{
  i2s_chan_config_t chan_cfg = I2S_CHANNEL_DEFAULT_CONFIG(I2S_NUM_0, I2S_ROLE_MASTER);
  ESP_ERROR_CHECK(i2s_new_channel(&chan_cfg, NULL, &rx_handle));

  i2s_pdm_rx_config_t pdm_rx_cfg = {
      .clk_cfg = I2S_PDM_RX_CLK_DEFAULT_CONFIG(SAMPLE_RATE),
      .slot_cfg = {
        .data_bit_width = I2S_DATA_BIT_WIDTH_16BIT,
        .slot_bit_width = I2S_SLOT_BIT_WIDTH_AUTO,
        .slot_mode = I2S_SLOT_MODE_MONO,
        .slot_mask = I2S_PDM_SLOT_RIGHT,
      },
      .gpio_cfg = {
          .clk = CONFIG_I2S_CLK_PIN,
          .din = CONFIG_I2S_DATA_IN_PIN,
          .invert_flags = {
              .clk_inv = false,
          },
      },
  };

  ESP_ERROR_CHECK(i2s_channel_init_pdm_rx_mode(rx_handle, &pdm_rx_cfg));
  ESP_ERROR_CHECK(i2s_channel_enable(rx_handle));
}

void setup(){
  pinMode(BUTTON_PIN, INPUT_PULLUP);
}

void loop() {
  /* Initialize the channel */

  bool buttonState = digitalRead(BUTTON_PIN);
  bool record = false;
  init_microphone();

  while (buttonState == false) {
        printf("Recording started !");
        // Read the RAW samples from the microphone
        if (i2s_channel_read(rx_handle, (char *)microphone_data, SAMPLE_SIZE, &bytes_read, 1000) == ESP_OK) {
            printf("[0] %d [1] %d [2] %d [3]%d ...\n", microphone_data[0], microphone_data[1], microphone_data[2], microphone_data[3]);
        } else {
            printf("Read Failed!\n");
        }
        vTaskDelay(100/portTICK_PERIOD_MS);
        buttonState = digitalRead(BUTTON_PIN);
        record = true;
  }

  if (record == true) {
    printf("Recording complete !");
  }

  i2s_channel_disable(rx_handle);
  i2s_del_channel(rx_handle);
}

The output doesn't appear to align with the PCM data format. Despite searching through the ESP-IDF documentation for a method to convert PDM to PCM, I couldn't find one. Here's what the output looks like on the serial monitor:

I'm having trouble understanding what I'm doing wrong. I would greatly appreciate any assistance you could provide 😃!

Thank you in advance.

mmackh commented 4 months ago

bump - is there any update on new audio drivers PDM -> PCM conversion?

xG3nesis commented 4 months ago

Additionnaly, I really think that i'm dealing with PDM because when i perform a manual PDM to PCM conversion by applying a low pass filter, decimation and high pass filter, i almost recover my audio in audacity.

I apologize, but I find the ESP-IDF documentation to be unclear regarding PDM Mode (RX) in version 5.1.2. Specifically, when they mention, "PDM (Pulse-density Modulation) mode for RX channel can receive PDM-format data and convert the data into PCM format" it's ambiguous. Does this imply that it has the capability to convert, or that it automatically performs the conversion? Do we need to initialize it somehow ?

xenpac commented 3 months ago

interesting.. The PDM input is automaticly converted to PCM in hardware, on the ESP32-V1 I2S-0. PDM Mic must be 16bit.

i am currently working on PDM Mic asto basic pdm example in 5.2.

There is a DC offset in the PCM output.

The Mics converted PCM Volume is extremely low. Is there any gain-setting ?

lin-k23 commented 2 weeks ago

@xenpac hi ,i am dealing with pdm mic too, the fig shows my raw data in audacity, with sample rate 16000, pcm format is signed 16-bit. i can see an obvious DC offset and failed to get correct result. how about the result of your problem? i am using esp_idf v5.2.2' example of pdm_rx.

espressif / esp-idf