search

esphome / issues

Issue Tracker for ESPHome
https://esphome.io/
292 stars 36 forks source link

M5 stack atom echo voice assistant simply does not work #5279

Open Nardol opened 11 months ago

Nardol commented 11 months ago

The problem

I have uploaded the following firmware, modified from the original from esphome/firmware (See below) on the example of YAML.

Firstly I am not sure why esp_adf is here because if I understand correctly, it is only for S3 box. But even without it, nothing works.

I set up wake word with a wyoming-porcupine1 docker container (on Podman) and its entity is discovered in Home Assistant. I selected a wake word in my language when setting up the Assistant, Porcupine1 is selected as wake word engine.

But nothing happens when saying the selected word. Not sure for the light as I am blind, but nothing is logged by running esphome logs except voice_assistant is waiting.

I tried to disable wake word switch. voice_assistant and microphone are switched off as expected but when pressing the button some seconds to be sure, nothing happens in the log, in fact nothing happens at all. When restarting the board it logs Assistant is waiting but if I talk, nothing happens at all and if switch it off by long-pressing the button, if I do it again voice_assistant stays idle.

Which version of ESPHome has the issue?

2023.12.3

What type of installation are you using?

pip

Which version of Home Assistant has the issue?

2023.12.3

What platform are you using?

ESP32-IDF

Board

m5stack-atom

Component causing the issue

voice_assistant

Example YAML snippet

---
esphome:
  name: enceinte-test
  friendly_name: "Enceinte test"
  project:
    name: m5stack.atom-echo-voice-assistant
    version: "1.0"
  min_version: 2023.11.1

esp32:
  board: m5stack-atom
  framework:
    type: esp-idf

logger:
  level: DEBUG
  logs:
    light: ERROR
    esp32_rmt_led_strip: ERROR
    binary_sensor: ERROR
    api: ERROR
    ota: ERROR
    audio: error
    mdns: ERROR
    wifi: ERROR
    wireguard: ERROR
    esp32.preferences: ERROR
    homeassistant.time: ERROR
    factory_reset.button: ERROR
    template.switch: ERROR
    gpio.binary_sensor: ERROR

api:
  encryption:
    key: !secret enceinte_test_encryption_key

ota:
  password: !secret enceinte_test_ota_pass

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_key
  #enable_btm: true
  #enable_rrm: true
  ap:
    password: !secret ap_key

button:
  - platform: factory_reset
    id: factory_reset_btn
    name: Factory reset

i2s_audio:
  i2s_lrclk_pin: GPIO33
  i2s_bclk_pin: GPIO19

microphone:
  - platform: i2s_audio
    id: echo_microphone
    i2s_din_pin: GPIO23
    adc_type: external
    pdm: true

speaker:
  - platform: i2s_audio
    id: echo_speaker
    i2s_dout_pin: GPIO22
    dac_type: external
    mode: mono

voice_assistant:
  id: va
  microphone: echo_microphone
  speaker: echo_speaker
  noise_suppression_level: 4
  auto_gain: 31dBFS
  volume_multiplier: 2.0
  vad_threshold: 3
  on_listening:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        effect: "Slow Pulse"
  on_stt_vad_end:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        effect: "Fast Pulse"
  on_tts_start:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        brightness: 100%
        effect: none
  on_end:
    - delay: 100ms
    - wait_until:
        not:
          speaker.is_playing:
    - script.execute: reset_led
  on_error:
    - light.turn_on:
        id: led
        red: 100%
        green: 0%
        blue: 0%
        brightness: 100%
        effect: none
    - delay: 1s
    - script.execute: reset_led
  on_client_connected:
    - if:
        condition:
          switch.is_on: use_wake_word
        then:
          - voice_assistant.start_continuous:
          - script.execute: reset_led
  on_client_disconnected:
    - if:
        condition:
          switch.is_on: use_wake_word
        then:
          - voice_assistant.stop:
          - light.turn_off: led

binary_sensor:
  - platform: gpio
    pin:
      number: GPIO39
      inverted: true
    name: Button
    disabled_by_default: true
    entity_category: diagnostic
    id: echo_button
    on_multi_click:
      - timing:
          - ON for at least 250ms
          - OFF for at least 50ms
        then:
          - if:
              condition:
                switch.is_off: use_wake_word
              then:
                - if:
                    condition: voice_assistant.is_running
                    then:
                      - voice_assistant.stop:
                      - script.execute: reset_led
                    else:
                      - voice_assistant.start:
              else:
                - voice_assistant.stop
                - delay: 1s
                - script.execute: reset_led
                - script.wait: reset_led
                - voice_assistant.start_continuous:
      - timing:
          - ON for at least 10s
        then:
          - button.press: factory_reset_btn

light:
  - platform: esp32_rmt_led_strip
    id: led
    name: None
    disabled_by_default: true
    entity_category: config
    pin: GPIO27
    default_transition_length: 0s
    chipset: SK6812
    num_leds: 1
    rgb_order: grb
    rmt_channel: 0
    effects:
      - pulse:
          name: "Slow Pulse"
          transition_length: 250ms
          update_interval: 250ms
          min_brightness: 50%
          max_brightness: 100%
      - pulse:
          name: "Fast Pulse"
          transition_length: 100ms
          update_interval: 100ms
          min_brightness: 50%
          max_brightness: 100%

script:
  - id: reset_led
    then:
      - if:
          condition:
            - switch.is_on: use_wake_word
            - switch.is_on: use_listen_light
          then:
            - light.turn_on:
                id: led
                red: 100%
                green: 89%
                blue: 71%
                brightness: 60%
                effect: none
          else:
            - light.turn_off: led

switch:
  - platform: template
    name: Use wake word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - lambda: id(va).set_use_wake_word(true);
      - if:
          condition:
            not:
              - voice_assistant.is_running
          then:
            - voice_assistant.start_continuous
      - script.execute: reset_led
    on_turn_off:
      - voice_assistant.stop
      - lambda: id(va).set_use_wake_word(false);
      - script.execute: reset_led
  - platform: template
    name: Use Listen Light
    id: use_listen_light
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - script.execute: reset_led
    on_turn_off:
      - script.execute: reset_led

network:
  enable_ipv6: true

time:
- platform: homeassistant
  timezone: "Europe/Paris"

wireguard:
  address: xxx.xxx.xxx.xxx
  netmask: 255.255.255.0
  private_key: !secret enceinte_test_wg_priv_key
  peer_endpoint: !secret wg_peer_endpoint
  peer_port: !secret wg_peer_port
  peer_public_key: !secret wg_peer_pub_key
  peer_persistent_keepalive: 25s
  peer_allowed_ips:
  - "xxx.xxx.xxx.0/24"

external_components:
  - source: github://pr#5230
    components:
      - esp_adf
    refresh: 0s

esp_adf:

Anything in the logs that might be useful for us?

Linking .pioenvs/enceinte-test/firmware.elf
/home/user/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: missing -
-end-group; added as last command line option

Additional information

I don't think it comes specifically from my board because the following firmware, using push to talk for assist, works:

---
esphome:
  name: enceinte-test
  friendly_name: "Enceinte test"
  project:
    name: m5stack.atom-echo
    version: "1.0"
  min_version: 2023.5.0

esp32:
  board: m5stack-atom
  framework:
    type: arduino

logger:
  level: ERROR
  logs:
    light: ERROR
    esp32_rmt_led_strip: ERROR
    binary_sensor: ERROR
    api: ERROR
    ota: ERROR
    audio: error
    mdns: ERROR
    wifi: ERROR

api:
  encryption:
    key: !secret enceinte_test_encryption_key

ota:
  password: !secret enceinte_test_ota_pass

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_key
  #enable_btm: true
  #enable_rrm: true
  ap:
    password: !secret ap_key

#captive_portal:

i2s_audio:
  i2s_lrclk_pin: GPIO33
  i2s_bclk_pin: GPIO19

microphone:
  - platform: i2s_audio
    id: echo_microphone
    i2s_din_pin: GPIO23
    adc_type: external
    pdm: true

voice_assistant:
  microphone: echo_microphone
  #speaker: speaker_out
  media_player: media_out
  noise_suppression_level: 4
  auto_gain: 31dBFS
  volume_multiplier: 2.0
  #vad_threshold: 3
  on_start:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        effect: none
  on_tts_start:
    - light.turn_on:
        id: led
        blue: 0%
        red: 0%
        green: 100%
        effect: none
  on_tts_end:
    #- media_player.play_media: !lambda return x;
    - light.turn_on:
        id: led
        blue: 0%
        red: 0%
        green: 100%
        effect: pulse
  on_end:
    - delay: 1s
    - wait_until:
        not:
          media_player.is_playing: media_out
    - light.turn_off: led
  on_error:
    - light.turn_on:
        id: led
        blue: 0%
        red: 100%
        green: 0%
        effect: none
    - delay: 1s
    - light.turn_off: led

binary_sensor:
  - platform: gpio
    pin:
      number: GPIO39
      inverted: true
    name: Button
    id: echo_button
    on_multi_click:
      - timing:
          - ON FOR AT MOST 350ms
          - OFF FOR AT LEAST 10ms
        then:
          - media_player.toggle: media_out
      - timing:
          - ON FOR AT LEAST 350ms
        then:
          - if:
              condition:
                media_player.is_playing: media_out
              then:
                - media_player.stop: media_out
          - voice_assistant.start:
      - timing:
          - ON FOR AT LEAST 350ms
          - OFF FOR AT LEAST 10ms
        then:
          - voice_assistant.stop:

#speaker:
  #- platform: i2s_audio
    #id: speaker_out
    #dac_type: external
    #i2s_dout_pin: GPIO22
    #mode: mono

media_player:
  - platform: i2s_audio
    id: media_out
    name: None
    dac_type: external
    i2s_dout_pin: GPIO22
    mode: mono

light:
  - platform: esp32_rmt_led_strip
    id: led
    name: None
    pin: GPIO27
    default_transition_length: 0s
    chipset: SK6812
    num_leds: 1
    rgb_order: grb
    rmt_channel: 0
    effects:
      - pulse:
          transition_length: 250ms
          update_interval: 250ms

network:
  enable_ipv6: true

text_sensor:
- platform: wifi_info
  ip_address:
    name: IP address

time:
- platform: homeassistant
  timezone: "Europe/Paris"

wireguard:
  address: xxx.xxx.xxx.xxx
  netmask: 255.255.255.0
  private_key: !secret enceinte_test_wg_priv_key
  peer_endpoint: !secret wg_peer_endpoint
  peer_port: !secret wg_peer_port
  peer_public_key: !secret wg_peer_pub_key
  peer_persistent_keepalive: 25s
  peer_allowed_ips:
  - "xxx.xx.xxx.0/24"

I would like to test wake word and even without it, having Assist working with the first firmware using a way or another and I really don't understand what I miss.

Tested with esphome 2023.11 and 2023.12 with the same results.

Nardol commented 11 months ago

It looks it is caused by too much components for not enough memory. With a minimal firmware without Wireguard, no API encryption, IPv6, no esp_adf and even no light management it is really better.

Pr0v0st commented 9 months ago

I have the exact same error:

/home/user/.platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: missing --end-group; added as last command line option

I'm using esphome version 2024.2.0 on linux mint.

the i2s speaker component doesn't seem to want to play nice with the esp-idf framework for me for some reason.

I'm using a MAX98357A. i2s microphone is working fine. I've only got a voice assistant, microphone and speaker components defined. there's no sound out of the speaker. If I add an rtttl component to test the speaker, I boot loop.

Were you able to get it working?

raress96 commented 4 months ago

Hello, I just installed the whole Voice thing. The text to speach works when testing from Voice Assistants, but the M5 Atom does nothing (used the ready made EspHome installation). I press the button and sometimes I speak and a red led lights up, but I couldn't get anything else to work...

I am using the Local Assist pipeline on a Raspberry Pi 4, 4 GB