Voice Assistant-Add support for the espressif esp32-korvo-v1.1

[rarroyo6]

Describe the problem you have/What new integration you would like

Add support for the espressif esp32-korvo-v1.1, see documentation here: https://github.com/espressif/esp-skainet/blob/master/docs/en/hw-reference/esp32/user-guide-esp32-korvo-v1.1.md Please describe your use case for this integration and alternatives you've tried:

This will provide another option for a voice assistant with all the needed features built-in. Additional context

This board is available, relatively inexpensive, and has a microphone array, leds, and speaker output.

[trip5]

Searching, got me intrigued... The Korvo appears to be It looks like it may be using the same codebase as the what Espressif calls ESP-BOX (ie the ESP32-S3-BOX)... which puts it under the scope of https://github.com/esphome/feature-requests/issues/2239 - doesn't it?

[huishizhao]

hope someone could study how to add it into ESPHOME. it's it include a ADC es7210 and es8311CODEC and LED control chip.

[rarroyo6]

Got one on order, will start playing with it as soon as I receive it.

[joey-90]

I managed to get an ESP32-s3-Korvo-1 wooing with the setup posted https://github.com/joey-90/ESP32-S3-Korvo-1---Voice-Assistant/blob/main/voiceassistant.yaml

I can't get wake word detection working at the moment though

[asve99]

I managed to get an ESP32-s3-Korvo-1 wooing with the setup posted https://github.com/joey-90/ESP32-S3-Korvo-1---Voice-Assistant/blob/main/voiceassistant.yaml

I can't get wake word detection working at the moment though

Thanks @joey-90 for this. I copied the contents of the yaml from your repo and tried to compile/load for my board but I get these msgs when compiling. The firmware loads into the board but none of the i2s components load/work. Have you seen these warnings/errors when compiling? I am running ESPHome 2023.10.3.

src/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp: In member function 'void esphome::i2s_audio::I2SAudioMicrophone::start_()':
src/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp:62:3: warning: missing initializer for member 'i2s_driver_config_t::chan_mask' [-Wmissing-field-initializers]
   };
   ^
src/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp:62:3: warning: missing initializer for member 'i2s_driver_config_t::total_chan' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp:62:3: warning: missing initializer for member 'i2s_driver_config_t::left_align' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp:62:3: warning: missing initializer for member 'i2s_driver_config_t::big_edin' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp:62:3: warning: missing initializer for member 'i2s_driver_config_t::bit_order_msb' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp:62:3: warning: missing initializer for member 'i2s_driver_config_t::skip_msk' [-Wmissing-field-initializers]
Compiling .pioenvs/esp32-voice-3/src/esphome/components/light/automation.o
src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp: In static member function 'static void esphome::i2s_audio::I2SAudioSpeaker::player_task(void*)':
src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp:56:3: warning: missing initializer for member 'i2s_driver_config_t::chan_mask' [-Wmissing-field-initializers]
   };
   ^
src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp:56:3: warning: missing initializer for member 'i2s_driver_config_t::total_chan' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp:56:3: warning: missing initializer for member 'i2s_driver_config_t::left_align' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp:56:3: warning: missing initializer for member 'i2s_driver_config_t::big_edin' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp:56:3: warning: missing initializer for member 'i2s_driver_config_t::bit_order_msb' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp:56:3: warning: missing initializer for member 'i2s_driver_config_t::skip_msk' [-Wmissing-field-initializers]

[netweaver1970]

I have the same compilation warnings and the same (non VAD working) behaviour with my Korvo-2. I used the S3-box yaml. Same warnings during compilation, looking good during the device esphome startup log (as far as I know) but also no wakeword detection. As per Jessie it should work very similar to the S3-BOX though ... https://discord.com/channels/429907082951524364/1163574334472863815/1166079003950588006

[joey-90]

I've updated the code posted in the repository as I have now got wake word detection working.

In team of compile errors, I did get some initially but was mostly caused by fat fingering repository names and not having the correct pin layout. I think the Korvo 2 and the non S3 Korvo have different Pin outs.

The key to getting it working mostly for me was using the esp32-s3-devkitc-1 board variant.

Have you got the code for this posted anywhere?

[asve99]

I've updated the code posted in the repository as I have now got wake word detection working.

In team of compile errors, I did get some initially but was mostly caused by fat fingering repository names and not having the correct pin layout. I think the Korvo 2 and the non S3 Korvo have different Pin outs.

The key to getting it working mostly for me was using the esp32-s3-devkitc-1 board variant.

Have you got the code for this posted anywhere?

Thanks, I'll dbl chk my board version, gpio pins.

[asve99]

I managed to get this working... As it turns out my esp32-korvo-v1.1 is not an esp32-s3, so I needed to change the board and the gpio pinouts. My board is this one: https://dl.espressif.com/dl/schematics/ESP32-KORVO_V1.1_schematics.pdf I am not 100% which board this should be in the esphome yaml, but I used the following:

esp32:
  board: esp32dev
  framework:
    type: esp-idf
    version: recommended
    sdkconfig_options:
      CONFIG_ESP32_DEFAULT_CPU_FREQ_240: "y"

Heres' the gpio pinouts I used:

i2c:
  sda: GPIO19 #GPIO1
  scl: GPIO32 #GPIO2
  scan: true
  frequency: 400kHz

output:
  - platform: gpio
    id: pa_ctrl
    pin: GPIO12 #GPIO38

i2s_audio:
  - id: codec
    i2s_lrclk_pin: GPIO22 #GPIO41 #ws
    i2s_bclk_pin: GPIO25 #GPIO40 #clk
    i2s_mclk_pin: GPIO0 #GPIO42
  - id: mic_adc
    i2s_lrclk_pin: GPIO26 #GPIO9 #ws
    i2s_bclk_pin: GPIO27 #GPIO10 #clk
    i2s_mclk_pin: GPIO0 #GPIO20

speaker:
  - platform: i2s_audio
    id: external_speaker
    dac_type: external
    i2s_audio_id: codec
    i2s_dout_pin: GPIO13 #GPIO39
    mode: mono

microphone:
  - platform: i2s_audio
    id: external_mic
    adc_type: external
    i2s_audio_id: mic_adc
    i2s_din_pin: GPIO36 #GPIO11
    pdm: false

light:
  - platform: esp32_rmt_led_strip
    id: led_ring
    name: "${friendly_name} Light"
    pin: GPIO33 #GPIO19

sensor:
  - id: button_adc
    platform: adc
    internal: true
    pin: 39 #8

The microphone, wakeword, LED all work fine. I currently dont have an external speaker to test that.

A big thank you to @joey-90 for the yaml.

[trip5]

A-ha. Is it the WROOM-32-based module here? https://www.aliexpress.com/item/1005002803964499.html

For about $30, it's a steal if this can actually act as a voice assistant.

[asve99]

A-ha. Is it the WROOM-32-based module here? https://www.aliexpress.com/item/1005002803964499.html

For about $30, it's a steal if this can actually act as a voice assistant.

Yes, that's the one I am using.

[rarroyo6]

That's the same board I ordered. Thanks to you guys for the work, I'll try it out as soon as I receive it. I'm thinking of 3D printing a small cylindrical case for it, with the board and microphones on top, and the speaker on the bottom. I'll post it after I get it designed.

[pyrodex]

I managed to get this working...

As it turns out my esp32-korvo-v1.1 is not an esp32-s3, so I needed to change the board and the gpio pinouts.

My board is this one: https://dl.espressif.com/dl/schematics/ESP32-KORVO_V1.1_schematics.pdf

I am not 100% which board this should be in the esphome yaml, but I used the following:

esp32:

  board: esp32dev

  framework:

    type: esp-idf

    version: recommended

    sdkconfig_options:

      CONFIG_ESP32_DEFAULT_CPU_FREQ_240: "y"

Heres' the gpio pinouts I used:

i2c:

  sda: GPIO19 #GPIO1

  scl: GPIO32 #GPIO2

  scan: true

  frequency: 400kHz

output:

  - platform: gpio

    id: pa_ctrl

    pin: GPIO12 #GPIO38

i2s_audio:

  - id: codec

    i2s_lrclk_pin: GPIO22 #GPIO41 #ws

    i2s_bclk_pin: GPIO25 #GPIO40 #clk

    i2s_mclk_pin: GPIO0 #GPIO42

  - id: mic_adc

    i2s_lrclk_pin: GPIO26 #GPIO9 #ws

    i2s_bclk_pin: GPIO27 #GPIO10 #clk

    i2s_mclk_pin: GPIO0 #GPIO20

speaker:

  - platform: i2s_audio

    id: external_speaker

    dac_type: external

    i2s_audio_id: codec

    i2s_dout_pin: GPIO13 #GPIO39

    mode: mono

microphone:

  - platform: i2s_audio

    id: external_mic

    adc_type: external

    i2s_audio_id: mic_adc

    i2s_din_pin: GPIO36 #GPIO11

    pdm: false

light:

  - platform: esp32_rmt_led_strip

    id: led_ring

    name: "${friendly_name} Light"

    pin: GPIO33 #GPIO19

sensor:

  - id: button_adc

    platform: adc

    internal: true

    pin: 39 #8

The microphone, wakeword, LED all work fine.

I currently dont have an external speaker to test that.

A big thank you to @joey-90 for the yaml.

Can you post somewhere the full YAML? I just ordered two of these, one for my test bench and one to create a nice setup once I can get a cool 3D printed case.

[asve99]

The microphone, wakeword, LED all work fine. I currently dont have an external speaker to test that. A big thank you to @joey-90 for the yaml.

Can you post somewhere the full YAML? I just ordered two of these, one for my test bench and one to create a nice setup once I can get a cool 3D printed case.

This is the full yaml I used. Please note I didn't manage to successfully test the speaker output. I am not sure if it was the speaker I was using or the yaml cfg.

substitutions:
  friendly_name: esp32-voice-3

esphome:
  name: esp32-voice-3
  platformio_options:
    board_build.flash_mode: dio
  on_boot:
    - priority: -100
      then:
        - wait_until: api.connected
        - delay: 1s
        - if:
            condition:
              switch.is_on: use_wake_word
            then:
              - voice_assistant.start_continuous:

esp32:
  board: esp-wrover-kit
  framework:
    #type: esp-idf
    type: arduino
    version: recommended

external_components:
  - source: github://rpatel3001/esphome@es8311
    components: [ es8311 ]
  - source: github://rpatel3001/esphome@es7210
    components: [ es7210 ]
  - source: github://pr#5230
    components:
      - esp_adf

# Enable logging
logger:

# Enable Home Assistant API
api:
  encryption:
    key: <REDACTED>

ota:
  password: <REDACTED>

wifi:
  ssid: <REDACTED>
  password: <REDACTED>
  use_address: <REDACTED>

i2c:
  sda: GPIO19 #GPIO1
  scl: GPIO32 #GPIO2
  scan: true
  frequency: 400kHz

es8311:
  address: 0x18

es7210:
  address: 0x40

output:
  - platform: gpio
    id: pa_ctrl
    pin: GPIO12 #GPIO38

i2s_audio:
  - id: codec
    i2s_lrclk_pin: GPIO22 #GPIO41 #ws
    i2s_bclk_pin: GPIO25 #GPIO40 #clk
    i2s_mclk_pin: GPIO0 #GPIO42
  - id: mic_adc
    i2s_lrclk_pin: GPIO26 #GPIO9 #ws
    i2s_bclk_pin: GPIO27 #GPIO10 #clk
    i2s_mclk_pin: GPIO0 #GPIO20

speaker:
  - platform: i2s_audio
    id: external_speaker
    dac_type: external
    i2s_audio_id: codec
    i2s_dout_pin: GPIO13 #GPIO39
    mode: mono

microphone:
  - platform: i2s_audio
    id: external_mic
    adc_type: external
    i2s_audio_id: mic_adc
    i2s_din_pin: GPIO36 #GPIO11
    pdm: false

voice_assistant:
  id: voice_asst
  microphone: external_mic
  speaker: external_speaker
  noise_suppression_level: 2
  auto_gain: 15dBFS
  volume_multiplier: 0.5
  use_wake_word: false
  on_listening:
    - light.turn_on:
        id: led_ring
        blue: 100%
        red: 0%
        green: 0%
        brightness: 100%
        effect: wakeword
  on_tts_start:
    - light.turn_on:
        id: led_ring
        blue: 0%
        red: 0%
        green: 100%
        brightness: 50%
        effect: pulse
  on_end:
    - delay: 100ms
    - wait_until:
        not:
          speaker.is_playing:
    - script.execute: reset_led
  on_error:
    - light.turn_on:
        id: led_ring
        blue: 0%
        red: 100%
        green: 0%
        brightness: 100%
        effect: none
    - delay: 1s
    - script.execute: reset_led
    - script.wait: reset_led
    - lambda: |-
        if (code == "wake-provider-missing" || code == "wake-engine-missing") {
          id(use_wake_word).turn_off();
        }

script:
  - id: reset_led
    then:
      - if:
          condition:
            switch.is_on: use_wake_word
          then:
            - light.turn_on:
                id: led_ring
                blue: 30%
                red: 0%
                green: 0%
                brightness: 25%
                effect: none
          else:
            - light.turn_off: led_ring

switch:
  - platform: template
    name: Use wake word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - lambda: id(voice_asst).set_use_wake_word(true);
      - if:
          condition:
            not:
              - voice_assistant.is_running
          then:
            - voice_assistant.start_continuous
      - script.execute: reset_led
    on_turn_off:
      - voice_assistant.stop
      - script.execute: reset_led

light:
  - platform: esp32_rmt_led_strip
    id: led_ring
    name: "${friendly_name} Light"
    pin: GPIO33 #GPIO19
    num_leds: 12
    rmt_channel: 0
    rgb_order: GRB
    chipset: ws2812
    default_transition_length: 0s
    effects:
      - pulse:
          name: "Pulse"
          transition_length: 0.5s
          update_interval: 0.5s
      - addressable_twinkle:
          name: "Working"
          twinkle_probability: 5%
          progress_interval: 4ms
      - addressable_color_wipe:
          name: "Wakeword"
          colors:
            - red: 0%
              green: 50%
              blue: 0%
              num_leds: 12
          add_led_interval: 40ms
          reverse: false

binary_sensor:
  - platform: template
    name: "${friendly_name} Volume Up"
    id: btn_volume_up
  - platform: template
    name: "${friendly_name} Volume Down"
    id: btn_volume_down
  - platform: template
    name: "${friendly_name} Set"
    id: btn_set
  - platform: template
    name: "${friendly_name} Play"
    id: btn_play
  - platform: template
    name: "${friendly_name} Mode"
    id: btn_mode
  - platform: template
    name: "${friendly_name} Record"
    id: btn_record
    on_press:
      - output.turn_on: pa_ctrl
      - voice_assistant.start:
      - light.turn_on:
          id: led_ring
          brightness: 100%
          effect: "Wakeword"
    on_release:
      - voice_assistant.stop:
      - output.turn_off: pa_ctrl
      - light.turn_off:
          id: led_ring

sensor:
  - id: button_adc
    platform: adc
    internal: true
    pin: 39 #8
    attenuation: 11db
    update_interval: 15ms
    filters:
      - median:
          window_size: 5
          send_every: 5
          send_first_at: 1
      - delta: 0.1
    on_value_range:
      - below: 0.55
        then:
          - binary_sensor.template.publish:
              id: btn_volume_up
              state: ON
      - above: 0.65
        below: 0.92
        then:
          - binary_sensor.template.publish:
              id: btn_volume_down
              state: ON
      - above: 1.02
        below: 1.33
        then:
          - binary_sensor.template.publish:
              id: btn_set
              state: ON
      - above: 1.43
        below: 1.77
        then:
          - binary_sensor.template.publish:
              id: btn_play
              state: ON
      - above: 1.87
        below: 2.15
        then:
          - binary_sensor.template.publish:
              id: btn_mode
              state: ON
      - above: 2.25
        below: 2.56
        then:
          - binary_sensor.template.publish:
              id: btn_record
              state: ON
      - above: 2.8
        then:
          - binary_sensor.template.publish:
              id: btn_volume_up
              state: OFF
          - binary_sensor.template.publish:
              id: btn_volume_down
              state: OFF
          - binary_sensor.template.publish:
              id: btn_set
              state: OFF
          - binary_sensor.template.publish:
              id: btn_play
              state: OFF
          - binary_sensor.template.publish:
              id: btn_mode
              state: OFF
          - binary_sensor.template.publish:
              id: btn_record
              state: OFF

[tobsch]

Great input folks! Just received my device and tested it out. Problem with @asve99's example: I receive a lot of errors telling me that the speaker buffer is full:

[21:25:52][W][voice_assistant:283]: Receive buffer full.
[21:25:52][W][voice_assistant:293]: Speaker buffer full.

I have a classic wired apple headphone connected to the jack. Any ideas?

[trip5]

Has anyone tried putting an enclosure on this yet? I'm also curious about the dimensions. The only spec I found said it's 88mm wide but no mention of the height - especially since it appears to be 2 boards strapped together... pretty hard to find 2 prefabricated plastic enclosures... Is 3D printing the best option?

[rarroyo6]

Just received my board today. If I get a chance this weekend to set it up, I'll work on designing a case that I have mind. 3D printing is the best option for this. I'll post my design when I finish it.

[rarroyo6]

Just tried to program the board. When I connect the USB, it does not get recognized as a serial port, when I connect a FTDI board to the pads on the PCB, it does not recognize the type of ESP chip. Did you guys have to do anything special to program it, or could it be that I got a defective board?

[asve99]

Just tried to program the board. When I connect the USB, it does not get recognized as a serial port, when I connect a FTDI board to the pads on the PCB, it does not recognize the type of ESP chip. Did you guys have to do anything special to program it, or could it be that I got a defective board?

Initially I thought I had the same issue with my board (not detected), but the issue was that I needed to connect power to the board (via the usb power connector) and connect your PC via the UART usb port on the board.

[rarroyo6]

Never mind, thanks for the help. Apparently the USB port on my computer had locked-up or something. I restarted my computer and it recognized it.

[abmantis]

@joey-90 thanks for your config! It works great except the speaker. Did you got that to work? On mine there is no sound (I'm using the headphone jack)

[rarroyo6]

I couldn't get the speaker to work either. I verified the I/O pins against the schematic, and they seem to match correctly. The following is what I get from the logs:

[21:42:04][D][voice_assistant:502]: Event Type: 8
[21:42:04][D][voice_assistant:572]: Response URL: "http://192.168.1.31:8123/api/tts_proxy/093290dcda7b2989879ced0ee701d7978ed0c34a_en-us_bcc7c1a994_tts.piper.raw"
[21:42:04][D][voice_assistant:395]: State changed from AWAITING_RESPONSE to STREAMING_RESPONSE
[21:42:04][D][voice_assistant:401]: Desired state set to STREAMING_RESPONSE
[21:42:04][D][i2s_audio.speaker:161]: Starting I2S Audio Speaker
[21:42:04][D][voice_assistant:502]: Event Type: 2
[21:42:04][D][voice_assistant:584]: Assist Pipeline ended
[21:42:04][D][i2s_audio.speaker:164]: Started I2S Audio Speaker
[21:42:04][D][voice_assistant:502]: Event Type: 98
[21:42:06][W][voice_assistant:293]: Speaker buffer full.
[21:42:06][W][voice_assistant:293]: Speaker buffer full.
[21:42:06][W][voice_assistant:293]: Speaker buffer full.
...
[21:42:07][W][voice_assistant:293]: Speaker buffer full.
[21:42:07][W][voice_assistant:293]: Speaker buffer full.
[21:42:07][D][voice_assistant:502]: Event Type: 99
[21:42:07][D][voice_assistant:395]: State changed from STREAMING_RESPONSE to RESPONSE_FINISHED
[21:42:07][D][voice_assistant:401]: Desired state set to IDLE
[21:42:07][D][voice_assistant:395]: State changed from RESPONSE_FINISHED to IDLE

[joey-90]

Apologies for the delay in getting back to everyone, its been a busy week or two with work etc.

I also can't get the speaker output to work, the pin out is definitely correct. The same pin config is used in the demo firmware from Espressif. I get similar error messages to what you are seeing. I've got some updated yaml config to upload where I've been playing with some of the values for the Voice assistant etc.

[rarroyo6]

Great! thank you, I would like to get this working. I'm also trying a couple of things on my end.

[pascalmtts]

I received mine today and can't get Assistant to work. When pressing the record button and releasing it nothing happens and I also don't receive any events in Home Asssitant. The Debug Page of my Voice Assistant says: There where no events in this run.

Besides of that I can see the event of the button press and the LED state change in Home Assistant from the device.

I used the same config as @asve99

Anyone with an idea why this is not working for me?

[pyrodex]

I received mine today and can't get Assistant to work. When pressing the record button and releasing it nothing happens and I also don't receive any events in Home Asssitant. The Debug Page of my Voice Assistant says: There where no events in this run.

Besides of that I can see the event of the button press and the LED state change in Home Assistant from the device.

I used the same config as @asve99

Anyone with an idea why this is not working for me?

The buttons aren't tied to any functions at the moment, you have to leverage a wake word from my personal experience.

[abmantis]

@pascalmtts see @joey-90 example on how to use the buttons. In his example voice is also working fine, but no sound output yet.

I am currently trying to add the Korvo1 board to ESP-ADF, which in turn should make it easy to get it into ESPHome.

[pascalmtts]

The button record is tied to a function in the template above:

  - platform: template
    name: "${friendly_name} Record"
    id: btn_record
    on_press:
      - output.turn_on: pa_ctrl
      - voice_assistant.start:
      - light.turn_on:
          id: led_ring
          brightness: 100%
          effect: "Wakeword"
    on_release:
      - voice_assistant.stop:
      - output.turn_off: pa_ctrl
      - light.turn_off:
          id: led_ring

As I did not get the wake word to work I tried using the record button, as it should start and stop the voice assistant.

I am currently trying to add the Korvo1 board to ESP-ADF, which in turn should make it easy to get it into ESPHome.

That's great! I don't know if you also have the Korvo1.1 on the list because I think these are 2 different boards. The Korvo1.1 is much cheaper then ESP32-S3-Korvo-1

Edit:

Here is a log output from ESPHome when pressing, holding and then releasing the rec button:

[15:51:54][D][binary_sensor:036]: 'ESP32 Korvo Record': Sending state ON
[15:51:54][D][voice_assistant:395]: State changed from IDLE to START_PIPELINE
[15:51:54][D][voice_assistant:401]: Desired state set to START_MICROPHONE
[15:51:54][D][light:036]: 'ESP32 Korvo Light' Setting:
[15:51:54][D][light:047]:   State: ON
[15:51:54][D][light:051]:   Brightness: 100%
[15:51:54][D][light:109]:   Effect: 'Wakeword'
[15:51:54][W][component:214]: Component adc.sensor took a long time for an operation (0.05 s).
[15:51:54][W][component:215]: Components should block for at most 20-30ms.
[15:51:54][D][voice_assistant:124]: microphone not running
[15:51:54][D][voice_assistant:206]: Requesting start...
[15:51:54][D][voice_assistant:395]: State changed from START_PIPELINE to STARTING_PIPELINE
[15:51:54][D][voice_assistant:124]: microphone not running
[15:51:54][D][voice_assistant:124]: microphone not running
[15:51:54][D][voice_assistant:124]: microphone not running
[15:51:54][D][voice_assistant:416]: Client started, streaming microphone
[15:51:54][D][voice_assistant:395]: State changed from STARTING_PIPELINE to START_MICROPHONE
[15:51:54][D][voice_assistant:401]: Desired state set to STREAMING_MICROPHONE
[15:51:54][D][voice_assistant:159]: Starting Microphone
[15:51:54][D][voice_assistant:395]: State changed from START_MICROPHONE to STARTING_MICROPHONE
[15:51:54][D][voice_assistant:395]: State changed from STARTING_MICROPHONE to STREAMING_MICROPHONE
[15:51:57][D][sensor:094]: 'button_adc': Sending state 3.13900 V with 2 decimals of accuracy
[15:51:57][D][binary_sensor:036]: 'ESP32 Korvo Volume Up': Sending state OFF
[15:51:57][D][binary_sensor:036]: 'ESP32 Korvo Volume Down': Sending state OFF
[15:51:57][D][binary_sensor:036]: 'ESP32 Korvo Set': Sending state OFF
[15:51:57][D][binary_sensor:036]: 'ESP32 Korvo Play': Sending state OFF
[15:51:57][D][binary_sensor:036]: 'ESP32 Korvo Mode': Sending state OFF
[15:51:57][D][binary_sensor:036]: 'ESP32 Korvo Record': Sending state OFF
[15:51:57][D][voice_assistant:495]: Signaling stop...
[15:51:57][D][voice_assistant:395]: State changed from STREAMING_MICROPHONE to STOP_MICROPHONE
[15:51:57][D][voice_assistant:401]: Desired state set to IDLE
[15:51:57][D][light:036]: 'ESP32 Korvo Light' Setting:
[15:51:57][D][light:047]:   State: OFF
[15:51:57][D][light:109]:   Effect: 'None'
[15:51:57][W][component:214]: Component adc.sensor took a long time for an operation (0.09 s).
[15:51:57][W][component:215]: Components should block for at most 20-30ms.
[15:51:57][D][voice_assistant:395]: State changed from STOP_MICROPHONE to STOPPING_MICROPHONE
[15:51:57][D][voice_assistant:395]: State changed from STOPPING_MICROPHONE to IDLE

[pascalmtts]

Alright I got it working. Problem was in my Assist configuration - after selecting Piper as TTS it is working now. Audio is also not working in my case, but I think we will find a solution for this in the future

[joey-90]

I've uploaded new YAML config having made some tweaks.

These are:

• Night light switch - enables a dim setting at night etc.
• Adjusted Voice assistant sensitivity - these are for a noisy room so YMMV
• Added a start up lighting routine.

Still no success with the sound output, but there is something to try. Will post over the weekend when I've had a chance to play.

Things to add:

• Detect WiFi connection lost and set indicator light
• Detect Home Assistant API status and set indicator lights
• Graft in changes from recent changes made by Jesserockz in Atom firmwares
• Look at using the SD card slot for local files

Please comment if there any suggestions and feel free to add pull requests if you see any errors or can add better code.

[rarroyo6]

I've uploaded new YAML config having made some tweaks.

These are:

* Night light switch - enables a dim setting at night etc.

* Adjusted Voice assistant sensitivity - these are for a noisy room so YMMV

* Added a start up lighting routine.

Still no success with the sound output, but there is something to try. Will post over the weekend when I've had a chance to play.

Things to add:

* Detect WiFi connection lost and set indicator light

* Detect Home Assistant API status and set indicator lights

* Graft in changes from recent changes made by Jesserockz in Atom firmwares

* Look at using the SD card slot for local files

Please comment if there any suggestions and feel free to add pull requests if you see any errors or can add better code.

Great work, thank you. I've been busy, so I haven't had too much time to play with this, but I'll download your YAML and try it out. Looking forward to your new changes.

[pyrodex]

I've uploaded new YAML config having made some tweaks.

These are:

• Night light switch - enables a dim setting at night etc.
• Adjusted Voice assistant sensitivity - these are for a noisy room so YMMV
• Added a start up lighting routine.

Still no success with the sound output, but there is something to try. Will post over the weekend when I've had a chance to play.

Things to add:

• Detect WiFi connection lost and set indicator light
• Detect Home Assistant API status and set indicator lights
• Graft in changes from recent changes made by Jesserockz in Atom firmwares
• Look at using the SD card slot for local files

Please comment if there any suggestions and feel free to add pull requests if you see any errors or can add better code.

I can't get your newer yaml to work on my v1.1 unit. I tried even serial flash.

I get this error:

Configuration does not match the platform of the connected device. Expected an ESP32S3 device.

[abmantis]

I was finally able to get both mic+speaker to work with esp-adf: https://github.com/abmantis/firmware/blob/korvo1/voice-assistant/esp32-s3-korvo1.yaml

Unfortunately, it is still a bit unstable and the sound output is cut off. But at least, it is progress!

[pascalmtts]

I was finally able to get both mic+speaker to work with esp-adf: https://github.com/abmantis/firmware/blob/korvo1/voice-assistant/esp32-s3-korvo1.yaml

Unfortunately, it is still a bit unstable and the sound output is cut off. But at least, it is progress!

Unfortunately this only works for the S3 Korvo1 right? We are using the ESP32 Korso 1.1 which isn't based on the s3

[joey-90]

I was finally able to get both mic+speaker to work with esp-adf: https://github.com/abmantis/firmware/blob/korvo1/voice-assistant/esp32-s3-korvo1.yaml

Unfortunately, it is still a bit unstable and the sound output is cut off. But at least, it is progress!

Unfortunately this only works for the S3 Korvo1 right? We are using the Espresso32 Korso 1.1 which isn't based on the s3

I had a play with the code provided by abmantis (thank you!) and was able to get working when added to my config. Weirdly it worked almost perfectly for day or so. I've made a change to a LED brightness and now there is no voice?

I also did some reading on the non s3 board variant and esp-add library is supported on the standard ESP32. It should be possible to create a custom board config at some point.

@abmantis - have you tried to access the SD card on the board at all yet?

[abmantis]

I was finally able to get both mic+speaker to work with esp-adf: https://github.com/abmantis/firmware/blob/korvo1/voice-assistant/esp32-s3-korvo1.yaml

Unfortunately, it is still a bit unstable and the sound output is cut off. But at least, it is progress!

Unfortunately this only works for the S3 Korvo1 right? We are using the Espresso32 Korso 1.1 which isn't based on the s3

I had a play with the code provided by abmantis (thank you!) and was able to get working when added to my config. Weirdly it worked almost perfectly for day or so. I've made a change to a LED brightness and now there is no voice?

Try completely powering it off and on again. Sometimes it seems to get "stuck".

@abmantis - have you tried to access the SD card on the board at all yet?

Only tried once and it was failing to initialize IIRC. I haven't spent much time looking into it yet, but I may have defined the wrong pinout on the board config.

[rarroyo6]

I was finally able to get both mic+speaker to work with esp-adf: https://github.com/abmantis/firmware/blob/korvo1/voice-assistant/esp32-s3-korvo1.yaml

Unfortunately, it is still a bit unstable and the sound output is cut off. But at least, it is progress!

Unfortunately this only works for the S3 Korvo1 right? We are using the Espresso32 Korso 1.1 which isn't based on the s3

I had a play with the code provided by abmantis (thank you!) and was able to get working when added to my config. Weirdly it worked almost perfectly for day or so. I've made a change to a LED brightness and now there is no voice?

Try completely powering it off and on again. Sometimes it seems to get "stuck".

@abmantis - have you tried to access the SD card on the board at all yet?

Only tried once and it was failing to initialize IIRC. I haven't spent much time looking into it yet, but I may have defined the wrong pinout on the board config.

I also had an issue with my board getting "stuck" every few hours. It losses connection to HA and pressing the "Rec" button doesn't respond. I have to press the reset button, or power it off/on. Is anybody else having this issue or could I have a defective board?

[pyrodex]

I was finally able to get both mic+speaker to work with esp-adf: https://github.com/abmantis/firmware/blob/korvo1/voice-assistant/esp32-s3-korvo1.yaml

Unfortunately, it is still a bit unstable and the sound output is cut off. But at least, it is progress!

I can't even get your YAML to compile... getting a few errors.

Compiling .pioenvs/esp32-s3-korvo-1/src/main.o
src/main.cpp: In function 'void setup()':
src/main.cpp:437:46: error: invalid new-expression of abstract class type 'esphome::esp_adf::ESPADFSpeaker'
   korvo_speaker = new esp_adf::ESPADFSpeaker();
                                              ^
In file included from src/esphome.h:39,
                 from src/main.cpp:3:
src/esphome/components/esp_adf/speaker/esp_adf_speaker.h:20:7: note:   because the following virtual functions are pure within 'esphome::esp_adf::ESPADFSpeaker':
 class ESPADFSpeaker : public ESPADFPipeline, public speaker::Speaker, public Component {
       ^~~~~~~~~~~~~
In file included from src/esphome/components/esp_adf/speaker/esp_adf_speaker.h:10,
                 from src/esphome.h:39,
                 from src/main.cpp:3:
src/esphome/components/speaker/speaker.h:21:16: note:   'virtual bool esphome::speaker::Speaker::has_buffered_data() const'
   virtual bool has_buffered_data() const = 0;
                ^~~~~~~~~~~~~~~~~

[rarroyo6]

I was finally able to get both mic+speaker to work with esp-adf: https://github.com/abmantis/firmware/blob/korvo1/voice-assistant/esp32-s3-korvo1.yaml Unfortunately, it is still a bit unstable and the sound output is cut off. But at least, it is progress!

I can't even get your YAML to compile... getting a few errors.

Compiling .pioenvs/esp32-s3-korvo-1/src/main.o
src/main.cpp: In function 'void setup()':
src/main.cpp:437:46: error: invalid new-expression of abstract class type 'esphome::esp_adf::ESPADFSpeaker'
   korvo_speaker = new esp_adf::ESPADFSpeaker();
                                              ^
In file included from src/esphome.h:39,
                 from src/main.cpp:3:
src/esphome/components/esp_adf/speaker/esp_adf_speaker.h:20:7: note:   because the following virtual functions are pure within 'esphome::esp_adf::ESPADFSpeaker':
 class ESPADFSpeaker : public ESPADFPipeline, public speaker::Speaker, public Component {
       ^~~~~~~~~~~~~
In file included from src/esphome/components/esp_adf/speaker/esp_adf_speaker.h:10,
                 from src/esphome.h:39,
                 from src/main.cpp:3:
src/esphome/components/speaker/speaker.h:21:16: note:     'virtual bool esphome::speaker::Speaker::has_buffered_data() const'
   virtual bool has_buffered_data() const = 0;
                ^~~~~~~~~~~~~~~~~

It was compiling fine, then I started getting the same error after I updated ESPHome to 2023.11.6. I don't remember which version I had before, but it was not 2023.11.5, it was at least 2 or 3 versions old.

[dwitgen]

Hello everyone, I used the same yaml as asve99 and it seems to work well except for no audio out and this working config/build is using the Arduino framework. I also was able to get one of the esp-idf framework yaml to work as well however that one keeps crashing after about 2 wakeups and has to be reset. For some reason speak recognition is very poor on the esp-idf build and although it does send audio to the speakers it is very hit or miss. Also, I had the same issue as rarryo6 and pyrodex with the esp-idf framework build and switched the external component of the esp-adf from:

external_components:

• source: github://abmantis/esphome@jesserockz-2023-284 components: esp_adf refresh: 0s

to:

external_components:

• source: github://pr#5230
  components: esp_adf refresh: 0s

This change compiled with no issues.

Unfortunately, I have only been using HA for about 6 months and ESPHome for maybe a month, so I do not have much insight however from my little experience the Arduino framework seems pretty stable, and I have had little issue with speech recognition just get no audio output.

Hopefully I will be able to provide more or be more of a contributor in the future for now I really hope someone finds a fix to the audio/speaker issues with this.

[rarroyo6]

Hello everyone, I used the same yaml as asve99 and it seems to work well except for no audio out and this working config/build is using the Arduino framework. I also was able to get one of the esp-idf framework yaml to work as well however that one keeps crashing after about 2 wakeups and has to be reset. For some reason speak recognition is very poor on the esp-idf build and although it does send audio to the speakers it is very hit or miss. Also, I had the same issue as rarryo6 and pyrodex with the esp-idf framework build and switched the external component of the esp-adf from:

external_components:

* source: github://abmantis/esphome@jesserockz-2023-284
  components: esp_adf
  refresh: 0s

to:

external_components:

* source: github://pr#5230
  components: esp_adf
  refresh: 0s

This change compiled with no issues.

Unfortunately, I have only been using HA for about 6 months and ESPHome for maybe a month, so I do not have much insight however from my little experience the Arduino framework seems pretty stable, and I have had little issue with speech recognition just get no audio output.

Hopefully I will be able to provide more or be more of a contributor in the future for now I really hope someone finds a fix to the audio/speaker issues with this.

Thank you, that seems to work. I haven't had much time to dedicate to this because of work, but hopefully after the new year I'll be able to dedicate more time to it. I would like to get this working, it looks like a nice board.

[schneipk]

Hey guys, good work and answers in this thread so far. Thank you all. I got things (my HA and ESP32-S3-Korvo-1 v5.0 NOT the ESP32-Korvo V1.1) to work with the work of @abmantis and the tweak to the PR by @rarroyo6. Only addition I had to make was framework: type: esp-idf version: 4.4.6 in the esp32 section of the ESPHome yaml File. I had to speak rather loud, and with a board equipped with 3 mics and AEC & other onboard sound quality features, so that made me want to tweak things. So I altered the settings in the yaml file section voice_assistant: id: voice_asst microphone: korvo_mic speaker: korvo_speaker noise_suppression_level: 4 auto_gain: 15dBFS Funnily I had to update my ESPHome installation as the compiler said I needed a newer version (although I successfully used the last one to compile/flash the "working" version). Now I can only speak one wake word and the lights immediately turn red and back to blue 1x. After that the wake word detection doesn't work anymore. Should I start a new thread? Any hints? Ongoing work? I would really like to leverage all the internal power the board brings to the game before sending the audio to HA. Thank you all & have a great evening!

[pbanj]

Hey guys, good work and answers in this thread so far. Thank you all. I got things (my HA and ESP32-S3-Korvo-1 v5.0 NOT the ESP32-Korvo V1.1) to work with the work of @abmantis and the tweak to the PR by @rarroyo6. Only addition I had to make was framework: type: esp-idf version: 4.4.6 in the esp32 section of the ESPHome yaml File. I had to speak rather loud, and with a board equipped with 3 mics and AEC & other onboard sound quality features, so that made me want to tweak things. So I altered the settings in the yaml file section voice_assistant: id: voice_asst microphone: korvo_mic speaker: korvo_speaker noise_suppression_level: 4 auto_gain: 15dBFS Funnily I had to update my ESPHome installation as the compiler said I needed a newer version (although I successfully used the last one to compile/flash the "working" version). Now I can only speak one wake word and the lights immediately turn red and back to blue 1x. After that the wake word detection doesn't work anymore. Should I start a new thread? Any hints? Ongoing work? I would really like to leverage all the internal power the board brings to the game before sending the audio to HA. Thank you all & have a great evening!

it's funny. i have the v1.1 but i got to the same state where i can do the wake word once then nothing. so im currently trying to figure that out. I'll add the working yaml once i get it wor4king and maybe it will help you

[pbanj]

ok so i have it working, it seems to keep working too after a while. increasing the delay time seems to have fixed it. i updated it to the release that just came out and it seems even more stable so i made it require it. i don't get audio out and i think it's because the v1.1 only supports Chinese according to the docs for it. i got rid of the gpio0 reuse error and the warning it shows.

esphome:
  name: bender
  friendly_name: bender
  min_version: 2023.12.8
  platformio_options:
    board_build.flash_mode: dio
  on_boot:
    - priority: -100
      then:
        - wait_until: api.connected
        - delay: 4s
        - if:
            condition:
              switch.is_on: use_wake_word
            then:
              - voice_assistant.start_continuous:

esp32:
  board: esp-wrover-kit
  framework:
    type: arduino
    version: recommended

external_components:
  - source: github://rpatel3001/esphome@es8311
    components: [ es8311 ]
  - source: github://rpatel3001/esphome@es7210
    components: [ es7210 ]
  - source: github://pr#5230
    components:
      - esp_adf

# Enable logging
logger:

# Enable Home Assistant API
api:
  encryption:
    key: !secret haapienc

ota:
  password: !secret haotaenc

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

i2c:
  sda: GPIO19 #GPIO1
  scl: GPIO32 #GPIO2
  scan: true
  frequency: 400kHz

es8311:
  address: 0x18

es7210:
  address: 0x40

output:
  - platform: gpio
    id: pa_ctrl
    pin: GPIO12 #GPIO38

i2s_audio:
  - id: codec
    i2s_lrclk_pin: GPIO22 #GPIO41 #ws
    i2s_bclk_pin: GPIO25 #GPIO40 #clk
    i2s_mclk_pin:
       number: GPIO0
       allow_other_uses: true
       ignore_strapping_warning: true
  - id: mic_adc
    i2s_lrclk_pin: GPIO26 #GPIO9 #ws
    i2s_bclk_pin: GPIO27 #GPIO10 #clk
    i2s_mclk_pin:
       number: GPIO0
       allow_other_uses: true
       ignore_strapping_warning: true

speaker:
  - platform: i2s_audio
    id: external_speaker
    dac_type: external
    i2s_audio_id: codec
    i2s_dout_pin: GPIO13 #GPIO39
    mode: mono

microphone:
  - platform: i2s_audio
    id: external_mic
    adc_type: external
    i2s_audio_id: mic_adc
    i2s_din_pin: GPIO36 #GPIO11
    pdm: false

voice_assistant:
  id: voice_asst
  microphone: external_mic
  speaker: external_speaker
  noise_suppression_level: 2
  auto_gain: 15dBFS
  volume_multiplier: 0.5
  use_wake_word: false
  on_listening:
    - light.turn_on:
        id: led_ring
        blue: 100%
        red: 0%
        green: 0%
        brightness: 100%
        effect: wakeword
  on_tts_start:
    - light.turn_on:
        id: led_ring
        blue: 0%
        red: 0%
        green: 100%
        brightness: 50%
        effect: pulse
  on_end:
    - delay: 100ms
    - wait_until:
        not:
          speaker.is_playing:
    - script.execute: reset_led
  on_error:
    - light.turn_on:
        id: led_ring
        blue: 0%
        red: 100%
        green: 0%
        brightness: 100%
        effect: none
    - delay: 1s
    - script.execute: reset_led
    - script.wait: reset_led
    - lambda: |-
        if (code == "wake-provider-missing" || code == "wake-engine-missing") {
          id(use_wake_word).turn_off();
        }

script:
  - id: reset_led
    then:
      - if:
          condition:
            switch.is_on: use_wake_word
          then:
            - light.turn_on:
                id: led_ring
                blue: 30%
                red: 0%
                green: 0%
                brightness: 25%
                effect: none
          else:
            - light.turn_off: led_ring

switch:
  - platform: template
    name: Use wake word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - lambda: id(voice_asst).set_use_wake_word(true);
      - if:
          condition:
            not:
              - voice_assistant.is_running
          then:
            - voice_assistant.start_continuous
      - script.execute: reset_led
    on_turn_off:
      - voice_assistant.stop
      - script.execute: reset_led

light:
  - platform: esp32_rmt_led_strip
    id: led_ring
    name: "${friendly_name} Light"
    pin: GPIO33 #GPIO19
    num_leds: 12
    rmt_channel: 0
    rgb_order: GRB
    chipset: ws2812
    default_transition_length: 0s
    effects:
      - pulse:
          name: "Pulse"
          transition_length: 0.5s
          update_interval: 0.5s
      - addressable_twinkle:
          name: "Working"
          twinkle_probability: 5%
          progress_interval: 4ms
      - addressable_color_wipe:
          name: "Wakeword"
          colors:
            - red: 0%
              green: 50%
              blue: 0%
              num_leds: 12
          add_led_interval: 40ms
          reverse: false

binary_sensor:
  - platform: template
    name: "${friendly_name} Volume Up"
    id: btn_volume_up
  - platform: template
    name: "${friendly_name} Volume Down"
    id: btn_volume_down
  - platform: template
    name: "${friendly_name} Set"
    id: btn_set
  - platform: template
    name: "${friendly_name} Play"
    id: btn_play
  - platform: template
    name: "${friendly_name} Mode"
    id: btn_mode
  - platform: template
    name: "${friendly_name} Record"
    id: btn_record
    on_press:
      - output.turn_on: pa_ctrl
      - voice_assistant.start:
      - light.turn_on:
          id: led_ring
          brightness: 100%
          effect: "Wakeword"
    on_release:
      - voice_assistant.stop:
      - output.turn_off: pa_ctrl
      - light.turn_off:
          id: led_ring

sensor:
  - id: button_adc
    platform: adc
    internal: true
    pin: 39 #8
    attenuation: 11db
    update_interval: 15ms
    filters:
      - median:
          window_size: 5
          send_every: 5
          send_first_at: 1
      - delta: 0.1
    on_value_range:
      - below: 0.55
        then:
          - binary_sensor.template.publish:
              id: btn_volume_up
              state: ON
      - above: 0.65
        below: 0.92
        then:
          - binary_sensor.template.publish:
              id: btn_volume_down
              state: ON
      - above: 1.02
        below: 1.33
        then:
          - binary_sensor.template.publish:
              id: btn_set
              state: ON
      - above: 1.43
        below: 1.77
        then:
          - binary_sensor.template.publish:
              id: btn_play
              state: ON
      - above: 1.87
        below: 2.15
        then:
          - binary_sensor.template.publish:
              id: btn_mode
              state: ON
      - above: 2.25
        below: 2.56
        then:
          - binary_sensor.template.publish:
              id: btn_record
              state: ON
      - above: 2.8
        then:
          - binary_sensor.template.publish:
              id: btn_volume_up
              state: OFF
          - binary_sensor.template.publish:
              id: btn_volume_down
              state: OFF
          - binary_sensor.template.publish:
              id: btn_set
              state: OFF
          - binary_sensor.template.publish:
              id: btn_play
              state: OFF
          - binary_sensor.template.publish:
              id: btn_mode
              state: OFF
          - binary_sensor.template.publish:
              id: btn_record
              state: OFF

[HA-TB303]

esp32-korvo 1.1, led works, mic works, wakeword works, but the speaker stays silent. Any suggestions?

substitutions:
  friendly_name: esp32-korvo

esphome:
  name: esp32-korvo-01
  min_version: 2023.12.8
  platformio_options:
    board_build.flash_mode: dio
  on_boot:
    - priority: -100
      then:
        - wait_until: api.connected
        - delay: 4s
        - if:
            condition:
              switch.is_on: use_wake_word
            then:
              - voice_assistant.start_continuous:

esp32:
  board: esp-wrover-kit
  framework:
    type: arduino
    version: recommended

external_components:
  - source: github://rpatel3001/esphome@es8311
    components: [ es8311 ]
  - source: github://rpatel3001/esphome@es7210
    components: [ es7210 ]
  - source: github://pr#5230
    components:
      - esp_adf

# Enable logging
logger:

# Enable Home Assistant API
api:
  encryption:
    key: !secret esphome_api_key

ota:
  password: !secret esphome_ota_password

wifi:
  ssid: !secret esphome_wifi_ssid
  password: !secret esphome_wifi_password

i2c:
  sda: GPIO19 #GPIO1
  scl: GPIO32 #GPIO2
  scan: true
  frequency: 400kHz

es8311:
  address: 0x18

es7210:
  address: 0x40

output:
  - platform: gpio
    id: pa_ctrl
    pin: GPIO12 #GPIO38

i2s_audio:
  - id: codec
    i2s_lrclk_pin: GPIO22 #GPIO41 #ws
    i2s_bclk_pin: GPIO25 #GPIO40 #clk
    i2s_mclk_pin:
       number: GPIO0
       allow_other_uses: true
       ignore_strapping_warning: true
  - id: mic_adc
    i2s_lrclk_pin: GPIO26 #GPIO9 #ws
    i2s_bclk_pin: GPIO27 #GPIO10 #clk
    i2s_mclk_pin:
       number: GPIO0
       allow_other_uses: true
       ignore_strapping_warning: true

speaker:
  - platform: i2s_audio
    id: external_speaker
    dac_type: external
    i2s_audio_id: codec
    i2s_dout_pin: GPIO13 #GPIO39
    mode: mono

microphone:
  - platform: i2s_audio
    id: external_mic
    adc_type: external
    i2s_audio_id: mic_adc
    i2s_din_pin: GPIO36 #GPIO11
    pdm: false

voice_assistant:
  id: voice_asst
  microphone: external_mic
  speaker: external_speaker
  noise_suppression_level: 2
  auto_gain: 15dBFS
  volume_multiplier: 0.5
  use_wake_word: false
  on_listening:
    - light.turn_on:
        id: led_ring
        blue: 100%
        red: 0%
        green: 0%
        brightness: 100%
        effect: wakeword
  on_tts_start:
    - light.turn_on:
        id: led_ring
        blue: 0%
        red: 0%
        green: 100%
        brightness: 50%
        effect: Working
  on_end:
    - delay: 100ms
    - wait_until:
        not:
          speaker.is_playing:
    - script.execute: reset_led
  on_error:
    - light.turn_on:
        id: led_ring
        blue: 0%
        red: 100%
        green: 0%
        brightness: 100%
        effect: none
    - delay: 1s
    - script.execute: reset_led
    - script.wait: reset_led
    - lambda: |-
        if (code == "wake-provider-missing" || code == "wake-engine-missing") {
          id(use_wake_word).turn_off();
        }

script:
  - id: reset_led
    then:
      - if:
          condition:
            switch.is_on: use_wake_word
          then:
            - light.turn_on:
                id: led_ring
                blue: 30%
                red: 0%
                green: 0%
                brightness: 25%
                effect: none
          else:
            - light.turn_off: led_ring

switch:
  - platform: template
    name: Use wake word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - lambda: id(voice_asst).set_use_wake_word(true);
      - if:
          condition:
            not:
              - voice_assistant.is_running
          then:
            - voice_assistant.start_continuous
      - script.execute: reset_led
    on_turn_off:
      - voice_assistant.stop
      - script.execute: reset_led

light:
  - platform: esp32_rmt_led_strip
    id: led_ring
    name: "${friendly_name} Light"
    pin: GPIO33 #GPIO19
    num_leds: 12
    rmt_channel: 0
    rgb_order: GRB
    chipset: ws2812
    default_transition_length: 0s
    effects:
      - pulse:
          name: "Pulse"
          transition_length: 0.5s
          update_interval: 0.5s
      - addressable_twinkle:
          name: "Working"
          twinkle_probability: 5%
          progress_interval: 4ms
      - addressable_color_wipe:
          name: "Wakeword"
          colors:
            - red: 0%
              green: 50%
              blue: 0%
              num_leds: 12
          add_led_interval: 40ms
          reverse: false

binary_sensor:
  - platform: template
    name: "${friendly_name} Volume Up"
    id: btn_volume_up
  - platform: template
    name: "${friendly_name} Volume Down"
    id: btn_volume_down
  - platform: template
    name: "${friendly_name} Set"
    id: btn_set
  - platform: template
    name: "${friendly_name} Play"
    id: btn_play
  - platform: template
    name: "${friendly_name} Mode"
    id: btn_mode
  - platform: template
    name: "${friendly_name} Record"
    id: btn_record
    on_press:
      - output.turn_on: pa_ctrl
      - voice_assistant.start:
      - light.turn_on:
          id: led_ring
          brightness: 100%
          effect: "Wakeword"
    on_release:
      - voice_assistant.stop:
      - output.turn_off: pa_ctrl
      - light.turn_off:
          id: led_ring

sensor:
  - id: button_adc
    platform: adc
    internal: true
    pin: 39 #8
    attenuation: 11db
    update_interval: 15ms
    filters:
      - median:
          window_size: 5
          send_every: 5
          send_first_at: 1
      - delta: 0.1
    on_value_range:
      - below: 0.55
        then:
          - binary_sensor.template.publish:
              id: btn_volume_up
              state: ON
      - above: 0.65
        below: 0.92
        then:
          - binary_sensor.template.publish:
              id: btn_volume_down
              state: ON
      - above: 1.02
        below: 1.33
        then:
          - binary_sensor.template.publish:
              id: btn_set
              state: ON
      - above: 1.43
        below: 1.77
        then:
          - binary_sensor.template.publish:
              id: btn_play
              state: ON
      - above: 1.87
        below: 2.15
        then:
          - binary_sensor.template.publish:
              id: btn_mode
              state: ON
      - above: 2.25
        below: 2.56
        then:
          - binary_sensor.template.publish:
              id: btn_record
              state: ON
      - above: 2.8
        then:
          - binary_sensor.template.publish:
              id: btn_volume_up
              state: OFF
          - binary_sensor.template.publish:
              id: btn_volume_down
              state: OFF
          - binary_sensor.template.publish:
              id: btn_set
              state: OFF
          - binary_sensor.template.publish:
              id: btn_play
              state: OFF
          - binary_sensor.template.publish:
              id: btn_mode
              state: OFF
          - binary_sensor.template.publish:
              id: btn_record
              state: OFF

[Instg1]

I managed to get an ESP32-s3-Korvo-1 wooing with the setup posted https://github.com/joey-90/ESP32-S3-Korvo-1---Voice-Assistant/blob/main/voiceassistant.yaml I can't get wake word detection working at the moment though

Thanks @joey-90 for this. I copied the contents of the yaml from your repo and tried to compile/load for my board but I get these msgs when compiling. The firmware loads into the board but none of the i2s components load/work. Have you seen these warnings/errors when compiling? I am running ESPHome 2023.10.3.

src/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp: In member function 'void esphome::i2s_audio::I2SAudioMicrophone::start_()':
src/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp:62:3: warning: missing initializer for member 'i2s_driver_config_t::chan_mask' [-Wmissing-field-initializers]
   };
   ^
src/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp:62:3: warning: missing initializer for member 'i2s_driver_config_t::total_chan' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp:62:3: warning: missing initializer for member 'i2s_driver_config_t::left_align' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp:62:3: warning: missing initializer for member 'i2s_driver_config_t::big_edin' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp:62:3: warning: missing initializer for member 'i2s_driver_config_t::bit_order_msb' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp:62:3: warning: missing initializer for member 'i2s_driver_config_t::skip_msk' [-Wmissing-field-initializers]
Compiling .pioenvs/esp32-voice-3/src/esphome/components/light/automation.o
src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp: In static member function 'static void esphome::i2s_audio::I2SAudioSpeaker::player_task(void*)':
src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp:56:3: warning: missing initializer for member 'i2s_driver_config_t::chan_mask' [-Wmissing-field-initializers]
   };
   ^
src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp:56:3: warning: missing initializer for member 'i2s_driver_config_t::total_chan' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp:56:3: warning: missing initializer for member 'i2s_driver_config_t::left_align' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp:56:3: warning: missing initializer for member 'i2s_driver_config_t::big_edin' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp:56:3: warning: missing initializer for member 'i2s_driver_config_t::bit_order_msb' [-Wmissing-field-initializers]
src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp:56:3: warning: missing initializer for member 'i2s_driver_config_t::skip_msk' [-Wmissing-field-initializers]

Hello, i am getting the exact same warnings with the esp32 c3 xiao from seeed studio,

No wake word detection. is there any solution on how to solve this problem, if you can help me please do it ill apreciated.

[trip5]

Most of this thread is about the ESP-Korvo-v1.1 - not the device you have. You're probably better off searching and/or making a new thread. Be sure to put the model number in the thread name so it's searchable.

@HA-TB303 Thanks for that YAML. After months of sitting on it, I finally got Piper, Whisper, OpenWakeWord, and Home Assistant working. (Had other projects to work until now.) This piqued my interest pretty hard and I have a few days off so I'll see if I can contribute a bit more to this thread.

Sidenote: There doesn't seem any one guide online to get the above components working for those of us running HA and the rest in docker. A bit of a pain in the butt... but joy. The wifey is gonna love this, even if there isn't a speaker.

Addition: Not sure about this link - don't own a 3D Printer and barely familiar with the whole process but looks like someone has already designed a pretty interesting looking case. Should fit the Korvo v1.1 but might fit others. Probably modifiable: https://www.printables.com/model/628182-entity-ai-assistant (I'll go 3rd party printing if I must, part of the wife-approval factor for sure.)

[trip5]

Been playing with this a few hours trying to make sense of https://dl.espressif.com/dl/schematics/ESP32-KORVO_V1.1_schematics.pdf versus the YAML as posted... and why we're not getting audio output.

I see this:

And I'm thinking that pa_ctrl means "power amplifier" which means GPIO12 can be used to detect headphones or not... or perhaps by pulling it high or low can turn on or off the amp. But looks like just the act of plugging something in here, it's turning off the amp... The bit with pa_ctrl in the "btn_record" section probably doesn't do anything useful...?

Also, I'm a little curious what ES7210_INT on GPIO34 does. INT usually means interrupt... perhaps a 'mute' pin for the speaker/codec? ESPHome is telling me I can't use it as as an output pin as such:

media_player:
  - platform: i2s_audio
    id: external_speaker
    dac_type: external
    i2s_audio_id: codec
    i2s_dout_pin: GPIO13
    mode: mono
    mute_pin:
      number: GPIO34
      inverted: true

Got the idea to try media_player instead of speaker from https://github.com/esphome/issues/issues/4976 - it compiles but doesn't help the issue. I'll dig through my gear to see if I can connect a speaker to the 2 pin-output to keep the amp on...? Just posting for now since I need a few days for family duties. Happy Lunar New Year!

[joie96]

Hello guys, do you know if esp32-korvo or esp32-s3-korvo support the audio front-end (AFE) algorithms from espressif in esphome? Is this something esphome could do? Especially acoustic echo cancellation (AEC) would be great.

[ALX-TH]

any luck to get speaker works on this board ?