ESPHome i2s_audio compatibility

[NonaSuomy]

Would it be plausible to make this compatible with https://github.com/esphome/esphome/tree/dev/esphome/components/i2s_audio

https://esphome.io/components/media_player/i2s_audio.html

https://rc.home-assistant.io/projects/thirteen-usd-voice-remote/

Example of another dac with i2c control for esphome. https://github.com/esphome/esphome/pull/3552

Example YAML setup for muse-luxe

https://github.com/esphome/media-players/blob/main/raspiaudio-muse-luxe.yaml

Microphone example YAML

https://github.com/esphome/media-players/blob/main/m5stack-atom-echo.yaml

https://github.com/esphome/firmware/blob/91f781174ada31bd124c9ab5ef6f642720c3cb60/voice-assistant/m5stack-atom-echo.yaml

Thank you.

[CelliesProjects]

As I understand it, the esphome mediaplayer is based on audioI2S. The wm8978 dac runs fine with audioI2S last time I checked so I think that could be done.

Correct me if I am wrong about all this.

[NonaSuomy]

Your understanding is correct! It is based on audioI2S

https://discord.gg/KhAMKrd

[NonaSuomy]

This is the echo YAML but pins swapped for the T-Audio 1.6 board. https://github.com/LilyGO/TTGO-TAudio

Original Atom Echo YAML:

https://github.com/esphome/firmware/blob/91f781174ada31bd124c9ab5ef6f642720c3cb60/voice-assistant/m5stack-atom-echo.yaml

YAML could look something like this:

# Substitute these words into the YAML below.
substitutions:
  name: "t-audio-001"
  friendly_name: "T-Audio-001"

# ESPHome Setup.
esphome:
  name: "${name}"
  friendly_name: "${friendly_name}"

# Board and Framework Setup.
esp32:
  board: esp-wrover-kit
  framework:
    type: arduino

# Enable logging.
logger:

# Enable and set up the Home Assistant API.
api:
  encryption:
    key: !secret encryption_key005

# Enable OTA updates.
ota:
  password: !secret ota_pass005

# Enable and set up WiFi.
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  use_address: !secret use_address_wifi005

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "${friendly_name} Fallback Hotspot"
    password: !secret fallbackhotspot005

# Enable captive portal if WiFi fails to connect.
captive_portal:

# Enable WebUI for direct control.
web_server:

# Configure the switch for restarting ESP32 from the WebUI.
switch:
  - platform: restart
    name: "${friendly_name} Restart"

# The improv_serial component in ESPHome implements the open Improv standard 
# for configuring Wi-Fi on an ESPHome device by using a serial connection to 
# the device, eg. USB.
improv_serial:

# Setup I2C pins for WM8978 communication.
i2c:
  sda: GPIO19
  scl: GPIO18

# Enable and set up the I2S audio pins.
i2s_audio:
  # Left / Right Clock or Word Select or Frame Sync.
  i2s_lrclk_pin: GPIO25
  # Bit Clock or Continuous Serial Clock [SCK].
  i2s_bclk_pin: GPIO33
  # Missing this pin setup?
  # Master Clock (typically 256 x LRCLK).
  #i2s_mclk_pin: 0

# Enable and set up the I2S microphone.
microphone:
  - platform: i2s_audio
    id: wm8978_microphone
    # Digital IN for the microphone.
    i2s_din_pin: GPIO27

# Enable and set up the media player for TTS files to play.
media_player:
  - platform: i2s_audio
    name: "${friendly_name} I2SAudio"
    id: wm8978_audio
    dac_type: external
    # I2S Digital Out for speaker(s).
    i2s_dout_pin: GPIO26
    mode: stereo

# Enable and set up the WM8978 DAC.
# Need a wm8978 component.
#wm8978:

# Enable voice assistant.
voice_assistant:
  microphone: wm8978_microphone
  on_start:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
  on_tts_start:
    - light.turn_on:
        id: led
        blue: 0%
        red: 0%
        green: 100%
  on_tts_end:
    - light.turn_on:
        id: led
        blue: 0%
        red: 0%
        green: 100%
    - media_player.play_media: !lambda return x;
  on_end:
    - delay: 1s
    - if:
        condition:
            media_player.is_playing:
        then:
          - wait_until:
              media_player.is_idle:
          - light.turn_off: led
        else:
          - light.turn_off: led
  on_error:
    - light.turn_on:
        id: led
        blue: 0%
        red: 100%
        green: 0%
    - delay: 1s
    - light.turn_off: led

# Enable and set up the microphone capture button.
binary_sensor:
  - platform: gpio
    pin:
      number: GPIO34
      inverted: true
    name: "${friendly_name} Button"
    disabled_by_default: true
    entity_category: diagnostic
    id: wm8978_button
    on_press:
      - voice_assistant.start:
    on_release:
      - voice_assistant.stop:

# Enable and set up the 19 addressable LED's on the T-Audio 1.6.
light:
  - platform: fastled_clockless
    id: led
    name: "${friendly_name} LED(s)"
    disabled_by_default: true
    entity_category: config
    pin: GPIO22
    default_transition_length: 0s
    chipset: WS2812B
    num_leds: 19
    rgb_order: GRB

It basically works (Button and LED haha) besides the wm8978 initialization code missing to turn off SOFTMUTE and enable the right and left audio channels.

Testing the button press (Doesn't detect audio from the microphone):

ESPHome version 2023.4.2 compiled on May  1 2023, 23:03:49
[23:04:23][D][api.connection:961]: Home Assistant 2023.5.0b5 (10.0.1.42): Connected successfully
[23:04:44][D][binary_sensor:036]: 'T-Audio-001 Button': Sending state ON
[23:04:44][D][voice_assistant:065]: Requesting start...
[23:04:44][D][voice_assistant:045]: Starting...
[23:04:44][D][voice_assistant:083]: Assist Pipeline running
[23:04:45][D][binary_sensor:036]: 'T-Audio-001 Button': Sending state OFF
[23:04:45][D][voice_assistant:073]: Signaling stop...
[23:04:53][E][voice_assistant:145]: Error: stt-no-text-recognized - No text recognized

Test sending a TTS wav file from Piper Docker in Home Assistant Docker with Wyoming Protocol (Doesn't output any sound).

[23:25:31][D][media_player:059]: 'T-Audio-001' - Setting
[23:25:31][D][media_player:066]:   Media URL: http://10.0.1.42:8123/api/tts_proxy/7b031c3d673c84477529b838252c3609e5be34b8_en-us_a877e2b3bf_tts.piper.wav

A bunch of docker-compose YAML I use for Home Assistant, Faster-Whisper, Piper, etc. https://github.com/NonaSuomy/nonasuomy.github.io/blob/master/_posts/2022-10-15-docker.md

[CelliesProjects]

Only possible problem I see is

  # Master Clock (typically 256 x LRCLK).
  #i2s_mclk_pin: 0

A wm8978 needs a clock signal and in a M5Stack Node it is pin 0. The required clock is provided by the esp32 I2S.

[NonaSuomy]

Looks like they are just hardcoding it?

https://github.com/esphome/esphome/pull/3552/commits/b1ad48c8240707eae18e4b7b9266c3c3ee01b239

namespace es8388 {

void ES8388Component::setup() {
  PIN_FUNC_SELECT(PERIPHS_IO_MUX_GPIO0_U, FUNC_GPIO0_CLK_OUT1);
  WRITE_PERI_REG(PIN_CTRL, READ_PERI_REG(PIN_CTRL) & 0xFFFFFFF0);

https://github.com/esphome/esphome/tree/fcf0761ba3ed1647b19ecbcb6ddcd7e2f5c0932c/esphome/components/es8388

[NonaSuomy]

Were you able to try it?

[ASIFKHAN95]

Compilation error: 'Audio' does not name a type error even when i compile its own examples