search

esphome / issues

Issue Tracker for ESPHome
https://esphome.io/
290 stars 36 forks source link

M5Stack Atom Echo no longer detects voice to start wakeword detection after a while #5038

Open rccoleman opened 1 year ago

rccoleman commented 1 year ago

The problem

There's been quite a bit of discussion about the M5 Atom Echo no longer responding to the programmed wakeword after "a while" (like overnight for me), but I don't see anything here and it seems related to the ESPHome build on this device. See the tail end of https://github.com/home-assistant/core/issues/101995 for observations and logs from the 2023.10.3 ESPHome build. In summary, the device seems to get stuck in WAITING_FOR_VAD mode until I either shout directly at it at close range, or sometimes even that's not enough. Without it transitioning from that state to recording voice, it won't detect the wakeword. Toggling the "Use wake word" switch in HA always restores normal behavior for me, and the device is otherwise responsive to commands from HA (like Use listen light) even when it's not detecting voice.

Which version of ESPHome has the issue?

2023.10.3

What type of installation are you using?

Docker

Which version of Home Assistant has the issue?

2023.11.0b0 currently, but also 2023.10.x

What platform are you using?

ESP32

Board

M5Stack Atom Echo

Component causing the issue

Voice Assistant

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

I'm using the default, unmodified voice assistant configuration for the Atom Echo, as first flashed via https://www.home-assistant.io/voice_control/thirteen-usd-voice-remote/ and then adopted by ESPHome and clean/build/install with updated ESPHome releases. There are various reports of updates fixing this issue (including from me), but I think it's just that it's not consistent and takes a while to reproduce. I started absolutely clean with the ESPHome 2023.10.3 container image, cleaned build files, and reinstalled the image on both devices yesterday and one was already exhibiting this behavior this morning.

zolakt commented 8 months ago

I have the same issue. The wake word works a few times, then it just stops (usually after a few minutes). Btw. it seems worse when it doesn't detect the speech correctly. I mean, it ends up in this frozen state sooner than it does when it recognizes a few commands successfully. Disabling and enabling wake word detection "fixes it".

I've read this whole thread, but honestly can't find a valid fix or a workaround (maybe I just missed it). Some change suggestions, but nothing definitive. So, is there a fix or a workaround for this?

Currently, I'm using the stupidest possible workaround... but it helps. I have a NodeRed automation running every 2 minutes (unless an assist is already in progress) which just toggles the wake word switch off and on again.

For what it's worth, initially I've flashed the firmware from the webinstaller, as per instructions in that 13$ voice assistant post. But I've noticed that the firmware was stuck at the same version for a few months. So, I flashed it again, manually from esphome addon, using the yaml esphome firmware. Currently it's running the 2023.12.9 firmware. It might have helped a little bit (not sure honestly), but it definitely didn't solve the problem.

Btw. I'm running HAOS on an RPI4

vtolstov commented 8 months ago

in my case my atom-echo does not detect any wakewords and assist_in_progress is always off

Neurocranium commented 8 months ago

I changed the input buffer read from mic and passed to the vad_process (from 32 to 64) and it solved in my case.

I tried this tweak as well and it truly seems to increase the reliability of the vad_process. Now the system is actually waiting for voice. Maybe even becoming a bit too insensitive, so I plan to do some experiments with different buffer length. Unfortunately, right now I have also a bunch of other unrelated issues going on with voice assistant on the Atom. :-( So, might be a longer process overall.

super-qua commented 8 months ago

From what I can see in my logs, the m5 and Piper / Wyoming Server lose the connection when this problem occurs. I don't know what causes them to lose the connection though.

Logger: homeassistant
Source: components/esphome/voice_assistant.py:354
First occurred: 17:38:56 (2 occurrences)
Last logged: 17:39:41

Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 1190, in sendto
    self._sock.sendto(data, addr)
    ^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'sendto'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/esphome/voice_assistant.py", line 354, in _send_tts
    self.transport.sendto(chunk, self.remote_addr)
  File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 1200, in sendto
    self._fatal_error(
  File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 867, in _fatal_error
    self._loop.call_exception_handler({
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'call_exception_handler'
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-32907' coro=<AsyncEventHandler.run() done, defined at /usr/local/lib/python3.11/dist-packages/wyoming/server.py:28> exception=ConnectionResetError('Connection lost')>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/wyoming/server.py", line 35, in run
    if not (await self.handle_event(event)):
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/wyoming_piper/handler.py", line 104, in handle_event
    await self.write_event(
  File "/usr/local/lib/python3.11/dist-packages/wyoming/server.py", line 26, in write_event
    await async_write_event(event, self.writer)
  File "/usr/local/lib/python3.11/dist-packages/wyoming/event.py", line 114, in async_write_event
    await writer.drain()
  File "/usr/lib/python3.11/asyncio/streams.py", line 378, in drain
    await self._protocol._drain_helper()
  File "/usr/lib/python3.11/asyncio/streams.py", line 167, in _drain_helper
    raise ConnectionResetError('Connection lost')
ConnectionResetError: Connection lost
michelcve commented 8 months ago

I have a similar problem, but the M5 echo seems to have crashed because of problems with the playback buffer. This is from the logging of the device:

ice_assistant:347]: Speaker buffer full, trying again next loop
[D][voice_assistant:347]: Speaker buffer full, trying again next loop
[D][voice_assistant:347]: Speaker buffer full, trying again next loop
[D][voice_assistant:347]: Speaker buffer full, trying again next loop
[D][voice_assistant:347]: Speaker buffer full, trying again next loop
[D][voice_assistant:347]: Speaker buffer full, trying again next loop
[D][voice_assistant:347]: Speaker buffer full, trying again next loop

It makes sense, since I sometimes notice that the playback of the response has gaps in it. And right now, it doesn't even respond to wake words anymore.

Firmware: 2023.12.9 (Feb 12 2024, 00:34:58) Hardware: 1.0

mrtnkhl commented 8 months ago

Seeing the same issue. Running three Atom Echo's connecting to a Home Assistant Yellow. After a while all Atom Echos start blinking red once every couple seconds and the HA instance on the Yellow becomes unresponsive. Does not even react to reboot request via web interface. Only way to get it working again is to pull power from the Yellow and all Atom Echos. Then the setup works for a random period of time before the same issue reoccurs.

ESPHome 2023.12.9 HA Core 2024.2.2 Supervisor 2024.01.1 HA OS 11.5 openWakeWord 1.8.2

Nothing relevant in the logs when the Yellow starts to freeze. Deactivating openWakeWord add on and Wyoming integration leads to stable HA Yellow. No other voice assistants than the Atom Echos mentioned above are connected to the Yellow.

Issue persists on Home Assistant Yellow with

After voice assistant commands work over a period of a few hours from enabling openWakeWord and Wyoming,

Purely a guess but maybe something is going wrong on the network level on the Yellow/HAOS, too many open connections initiated from the M5 Atoms make the network unresponsive and stall the kernel.

Only thing that helps is a hard reboot (pull power) and deactivation of openWakeWord and Wyoming. System is stable without those components running.

skyw33 commented 5 months ago

I did 2 things to get my atom echo to work. I don't know what exactly the root cause is but maybe this will help someone?

I run esphome using docker. First, I removed the ESPHOME_DASHBOARD_USE_PING=true environmental parameter for ESPHome running on Docker. ESPHOME_DASHBOARD_USE_PING disables mDNS.

Once I did this, the esphome dashboard showed a 2nd new instance of the atom echo device and the new instance had an "adopt" button. I first copied the yaml from the old instance and saved it to a text file. Then I removed the old instance and adopted the new instance using the auto-generated yaml.

You shouldn't need to use it, but this is what the new yaml looks like for reference:

substitutions:
  name: atom-echo-kitchen-asdf
  friendly_name: Atom Echo Kitchen asdf
packages:
  m5stack.atom-echo-voice-assistant: github://esphome/firmware/voice-assistant/m5stack-atom-echo.yaml@main
esphome:
  name: ${name}
  name_add_mac_suffix: false
  friendly_name: ${friendly_name}
api:
  encryption:
    key: asdfasdfasdfasdfasdf
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

And this is what the old file that I replaced looked like:

---
substitutions:
  name: "atom-echo-kitchen"
  friendly_name: "Atom Echo Kitchen"

esphome:
  name: "${name}"
  friendly_name: "${friendly_name}"
  name_add_mac_suffix: true
  project:
    name: m5stack.atom-echo-voice-assistant
    version: "1.0"
  min_version: 2023.10.0
  on_boot:
    - priority: -100
      then:
        - wait_until: api.connected
        - delay: 1s
        - if:
            condition:
              switch.is_on: use_wake_word
            then:
              - voice_assistant.start_continuous:

esp32:
  board: m5stack-atom
  framework:
    type: esp-idf

logger:
api:
ota:

dashboard_import:
  package_import_url: github://esphome/firmware/voice-assistant/m5stack-atom-echo.yaml@main

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  use_address: 192.168.1.43
  manual_ip:
    static_ip: 192.168.1.43
    gateway: 192.168.1.1
    subnet: 255.255.255.0
  ap:

improv_serial:

i2s_audio:
  i2s_lrclk_pin: GPIO33
  i2s_bclk_pin: GPIO19

microphone:
  - platform: i2s_audio
    id: echo_microphone
    i2s_din_pin: GPIO23
    adc_type: external
    pdm: true

speaker:
  - platform: i2s_audio
    id: echo_speaker
    i2s_dout_pin: GPIO22
    dac_type: external
    mode: mono

voice_assistant:
  id: va
  microphone: echo_microphone
  speaker: echo_speaker
  noise_suppression_level: 2
  auto_gain: 31dBFS
  volume_multiplier: 2.0
  vad_threshold: 3
  on_listening:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        brightness: 100%
        effect: pulse
  on_tts_start:
    - light.turn_on:
        id: led
        blue: 0%
        red: 0%
        green: 100%
        brightness: 100%
        effect: pulse
  on_end:
    - delay: 100ms
    - wait_until:
        not:
          speaker.is_playing:
    - script.execute: reset_led
  on_error:
    - light.turn_on:
        id: led
        blue: 0%
        red: 100%
        green: 0%
        brightness: 100%
        effect: none
    - delay: 1s
    - script.execute: reset_led
    - script.wait: reset_led
    - lambda: |-
        if (code == "wake-provider-missing" || code == "wake-engine-missing") {
          id(use_wake_word).turn_off();
        }

binary_sensor:
  - platform: gpio
    pin:
      number: GPIO39
      inverted: true
    name: Button
    disabled_by_default: true
    entity_category: diagnostic
    id: echo_button
    on_click:
      - if:
          condition:
            switch.is_off: use_wake_word
          then:
            - if:
                condition: voice_assistant.is_running
                then:
                  - voice_assistant.stop:
                  - script.execute: reset_led
                else:
                  - voice_assistant.start:
          else:
            - voice_assistant.stop
            - delay: 1s
            - script.execute: reset_led
            - script.wait: reset_led
            - voice_assistant.start_continuous:

light:
  - platform: esp32_rmt_led_strip
    id: led
    name: None
    disabled_by_default: true
    entity_category: config
    pin: GPIO27
    default_transition_length: 0s
    chipset: SK6812
    num_leds: 1
    rgb_order: grb
    rmt_channel: 0
    effects:
      - pulse:
          transition_length: 250ms
          update_interval: 250ms

script:
  - id: reset_led
    then:
      - if:
          condition:
            - switch.is_on: use_wake_word
            - switch.is_on: use_listen_light
          then:
            - light.turn_on:
                id: led
                blue: 100%
                red: 100%
                green: 0%
                brightness: 100%
                effect: none
          else:
            - light.turn_off: led

switch:
  - platform: template
    name: Use wake word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - lambda: id(va).set_use_wake_word(true);
      - if:
          condition:
            not:
              - voice_assistant.is_running
          then:
            - voice_assistant.start_continuous
      - script.execute: reset_led
    on_turn_off:
      - voice_assistant.stop
      - lambda: id(va).set_use_wake_word(false);
      - script.execute: reset_led
  - platform: template
    name: Use Listen Light
    id: use_listen_light
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - script.execute: reset_led
    on_turn_off:
      - script.execute: reset_led

external_components:
  - source: github://pr#5230
    components:
      - esp_adf
    refresh: 0s

esp_adf:

After doing these 2 things my atom echo has been rock solid.

big-bigchen commented 3 months ago

Hi, After updating the ESPHome to 2024.7 nnd using the latest config from here (https://github.com/esphome/firmware/blob/main/voice-assistant/m5stack-atom-echo.yaml) my echo is not activated by default. Only if I tick on it (produce a loud input) the wake word listening is started. I tuned down these parameters already:

voice_assistant:
  id: va
  microphone: echo_microphone
  speaker: echo_speaker
  noise_suppression_level: 0
  auto_gain: 0dBFS
  volume_multiplier: 4.0
  vad_threshold: 0

but no way to trigger the wake word detection without shouting at it... :-( Before the update to latest ESPHome and older Echo Atom firmware it worked quite well... Any ideas what to change? Thanks!

skyw33 commented 3 months ago

After installing the yaml from the below link, my atom echo has been running for 3 days straight -- no restarts. https://github.com/esphome/firmware/issues/227#issuecomment-2243440334

It uses the newly released micro wake word v2.