Open smartmatic opened 10 months ago
Hey there @ottowinter, @jesserockz, @kbx81, @bdraco, mind taking a look at this issue as it has been labeled with an integration (esphome
) you are listed as a code owner for? Thanks!
(message by CodeOwnersMention)
esphome documentation esphome source (message by IssueLinks)
Got the same issue after the update to use wake word I also get this in the voice assistant debug:
I see the same effect. Sometimes it crash and will not come back. I can fix it by turn of the Use Wake Word
switch and turn it on again.
Update: After upgrading to 2023.10.3 the turn off and on of Use Wake Word
does not fix the issue and is fully broken. Restart the openWakeWord addon 2 times fix the issue.
I am on HomeAssistant 2023.10.2 and ESPHome 2023.10.0b2 and am also seeing this issue. After a few times the atom echo seems to be confused (or HA is), and this can be solved by turning off wake-word and turning it on again.
Just rebooting the atom echo does not solve the issue for me, I always need to flick the wake-word button as well.
I have a debug dump of an assist pipeline that always causes the issue for me. I tried different wakewords and different voices. When it fails, it doesn't play the TTS on the atom, and then the atom doesn't respond to new wakewords.
stage: done
run:
pipeline: 01hcm8sjepme05hgfe8bemnp71
language: nl
events:
- type: run-start
data:
pipeline: 01hcm8sjepme05hgfe8bemnp71
language: nl
timestamp: "2023-10-14T20:30:59.086721+00:00"
- type: wake_word-start
data:
entity_id: wake_word.openwakeword
metadata:
format: wav
codec: pcm
bit_rate: 16
sample_rate: 16000
channel: 1
timeout: 5
timestamp: "2023-10-14T20:30:59.087441+00:00"
- type: wake_word-end
data:
wake_word_output:
wake_word_id: hey_mycroft_v0.1
timestamp: 1010
timestamp: "2023-10-14T20:31:00.971991+00:00"
- type: stt-start
data:
engine: cloud
metadata:
language: nl-NL
format: wav
codec: pcm
bit_rate: 16
sample_rate: 16000
channel: 1
timestamp: "2023-10-14T20:31:00.973212+00:00"
- type: stt-vad-start
data:
timestamp: 1670
timestamp: "2023-10-14T20:31:02.254087+00:00"
- type: stt-vad-end
data:
timestamp: 2630
timestamp: "2023-10-14T20:31:04.172964+00:00"
- type: stt-end
data:
stt_output:
text: Welk licht is aan in voorkamer?
timestamp: "2023-10-14T20:31:04.378965+00:00"
- type: intent-start
data:
engine: homeassistant
language: nl
intent_input: Welk licht is aan in voorkamer?
conversation_id: null
device_id: 660f1901da3c17af3b8e43ff97e57936
timestamp: "2023-10-14T20:31:04.379366+00:00"
- type: intent-end
data:
intent_output:
response:
speech:
plain:
speech: Erker 1 Light, Erker 2 Light, Silicon Labs EZSP Erker en nog 2
extra_data: null
card: {}
language: nl
response_type: query_answer
data:
targets: []
success:
- name: Voorkamer
type: area
id: 4554badea03a4da0b6e5e90c76e70685
- name: Voorkamer staande lamp Light
type: entity
id: light.voorkamer_staande_lamp_light
- name: Erker 1 Light
type: entity
id: light.erker_1_light_2
- name: Erker 2 Light
type: entity
id: light.erker_2_light_2
- name: Silicon Labs EZSP Erker
type: entity
id: light.silicon_labs_ezsp_erker
- name: Voorkamerlampen
type: entity
id: light.silicon_labs_ezsp_voorkamer
failed: []
conversation_id: null
timestamp: "2023-10-14T20:31:04.613753+00:00"
- type: tts-start
data:
engine: cloud
language: nl-NL
voice: FennaNeural
tts_input: Erker 1 Light, Erker 2 Light, Silicon Labs EZSP Erker en nog 2
timestamp: "2023-10-14T20:31:04.614193+00:00"
- type: tts-end
data:
tts_output:
media_id: >-
media-source://tts/cloud?message=Erker+1+Light,+Erker+2+Light,+Silicon+Labs+EZSP+Erker+en+nog+2&language=nl-NL&voice=FennaNeural&audio_output=raw
url: >-
/api/tts_proxy/1ae6716acf00c609bdb92c44285bb217b7326add_nl-nl_594a6dc511_cloud.raw
mime_type: audio/mpeg
timestamp: "2023-10-14T20:31:04.615234+00:00"
- type: run-end
data: null
timestamp: "2023-10-14T20:31:04.616059+00:00"
wake_word:
entity_id: wake_word.openwakeword
metadata:
format: wav
codec: pcm
bit_rate: 16
sample_rate: 16000
channel: 1
timeout: 5
done: true
wake_word_output:
wake_word_id: hey_mycroft_v0.1
timestamp: 1010
stt:
engine: cloud
metadata:
language: nl-NL
format: wav
codec: pcm
bit_rate: 16
sample_rate: 16000
channel: 1
done: true
stt_output:
text: Welk licht is aan in voorkamer?
intent:
engine: homeassistant
language: nl
intent_input: Welk licht is aan in voorkamer?
conversation_id: null
device_id: 660f1901da3c17af3b8e43ff97e57936
done: true
intent_output:
response:
speech:
plain:
speech: Erker 1 Light, Erker 2 Light, Silicon Labs EZSP Erker en nog 2
extra_data: null
card: {}
language: nl
response_type: query_answer
data:
targets: []
success:
- name: Voorkamer
type: area
id: 4554badea03a4da0b6e5e90c76e70685
- name: Voorkamer staande lamp Light
type: entity
id: light.voorkamer_staande_lamp_light
- name: Erker 1 Light
type: entity
id: light.erker_1_light_2
- name: Erker 2 Light
type: entity
id: light.erker_2_light_2
- name: Silicon Labs EZSP Erker
type: entity
id: light.silicon_labs_ezsp_erker
- name: Voorkamerlampen
type: entity
id: light.silicon_labs_ezsp_voorkamer
failed: []
conversation_id: null
tts:
engine: cloud
language: nl-NL
voice: FennaNeural
tts_input: Erker 1 Light, Erker 2 Light, Silicon Labs EZSP Erker en nog 2
done: true
tts_output:
media_id: >-
media-source://tts/cloud?message=Erker+1+Light,+Erker+2+Light,+Silicon+Labs+EZSP+Erker+en+nog+2&language=nl-NL&voice=FennaNeural&audio_output=raw
url: >-
/api/tts_proxy/1ae6716acf00c609bdb92c44285bb217b7326add_nl-nl_594a6dc511_cloud.raw
mime_type: audio/mpeg
Last loglines on the ESPhome log view of the atom are:
[22:31:00][D][voice_assistant:414]: Event Type: 10
[22:31:00][D][voice_assistant:423]: Wake word detected
[22:31:01][D][voice_assistant:414]: Event Type: 3
[22:31:01][D][voice_assistant:428]: STT Started
[22:31:01][D][light:036]: 'M5Stack Atom Echo 80b908' Setting:
[22:31:01][D][light:051]: Brightness: 100%
[22:31:01][D][light:059]: Red: 0%, Green: 0%, Blue: 100%
[22:31:01][D][light:109]: Effect: 'Pulse'
[22:31:04][D][voice_assistant:414]: Event Type: 4
[22:31:04][D][voice_assistant:313]: State changed from 7 to 8
[22:31:04][D][voice_assistant:319]: Desired state set to 10
[22:31:04][D][voice_assistant:443]: Speech recognised as: "Welk licht is aan in voorkamer?"
[22:31:04][D][voice_assistant:313]: State changed from 8 to 9
[22:31:04][D][esp-idf:000]: I (4185493) I2S: DMA queue destroyed
[22:31:04][D][voice_assistant:414]: Event Type: 5
[22:31:04][D][voice_assistant:535]: Unhandled event type: 5
[22:31:04][D][voice_assistant:313]: State changed from 9 to 10
[22:31:04][D][voice_assistant:414]: Event Type: 6
[22:31:04][D][voice_assistant:414]: Event Type: 7
[22:31:04][D][voice_assistant:466]: Response: "Erker 1 Light, Erker 2 Light, Silicon Labs EZSP Erker en nog 2"
[22:31:04][D][light:036]: 'M5Stack Atom Echo 80b908' Setting:
[22:31:04][D][light:051]: Brightness: 100%
[22:31:04][D][light:059]: Red: 0%, Green: 100%, Blue: 0%
[22:31:04][D][voice_assistant:414]: Event Type: 8
[22:31:04][D][voice_assistant:484]: Response URL: "http://192.168.X.Y:8123/api/tts_proxy/1ae6716acf00c609bdb92c44285bb217b7326add_nl-nl_594a6dc511_cloud.raw"
[22:31:04][D][voice_assistant:313]: State changed from 10 to 11
[22:31:04][D][voice_assistant:319]: Desired state set to 11
[22:31:04][D][voice_assistant:414]: Event Type: 2
[22:31:04][D][voice_assistant:496]: Assist Pipeline ended
[22:31:04][D][esp-idf:000]: I (4185878) I2S: DMA queue destroyed
[22:31:04][D][light:036]: 'M5Stack Atom Echo 80b908' Setting:
[22:31:04][D][light:051]: Brightness: 100%
[22:31:04][D][light:059]: Red: 100%, Green: 0%, Blue: 100%
[22:31:04][D][light:109]: Effect: 'None'
[22:31:04][D][voice_assistant:313]: State changed from 11 to 0
[22:31:04][D][voice_assistant:319]: Desired state set to 0
[22:31:04][D][voice_assistant:313]: State changed from 0 to 1
[22:31:04][D][voice_assistant:319]: Desired state set to 3
[22:31:04][D][voice_assistant:159]: Starting Microphone
[22:31:04][D][voice_assistant:313]: State changed from 1 to 2
[22:31:04][D][esp-idf:000]: I (4186009) I2S: DMA Malloc info, datalen=blocksize=1024, dma_buf_count=4
[22:31:04][D][voice_assistant:313]: State changed from 2 to 3
[22:31:04][D][voice_assistant:176]: Waiting for speech...
[22:31:04][D][voice_assistant:313]: State changed from 3 to 4
[22:31:05][D][voice_assistant:189]: VAD detected speech
[22:31:05][D][voice_assistant:313]: State changed from 4 to 5
[22:31:05][D][voice_assistant:319]: Desired state set to 7
[22:31:05][D][voice_assistant:206]: Requesting start...
[22:31:05][D][voice_assistant:313]: State changed from 5 to 6
[22:31:05][D][voice_assistant:334]: Client started, streaming microphone
[22:31:05][D][voice_assistant:313]: State changed from 6 to 7
[22:31:05][D][voice_assistant:319]: Desired state set to 7
[22:31:05][D][voice_assistant:414]: Event Type: 1
[22:31:05][D][voice_assistant:417]: Assist Pipeline running
[22:31:05][D][voice_assistant:414]: Event Type: 9
And then the whole setup doesn't function anymore, until I disable/enable 'use wakeword' on the device in HA.
It seems the latest ESP home update fixed the issue on my setup.
Edit: I had one issue again where I need to restart the openWakeWord addon to get it back on track.
Feel free to close the issue if you think this is solved for you :)
I upgraded to latest ESPHome version and still have to turn wake word switch on and off to get a response.
I'm fully upgraded and seeing this as well. I.e. the atom echo wake word works a few times, but then stops responding after a period of time. I can revive it by pressing the top button or holding it down for a while and then trying to speak again.
Ok, I was on ESPHome 10.1 and I just upgraded to 10.3. The ESPHome dashboard then said that there was an update available so I applied the upgrade and things are working so far. I'll update here if I have any more issues.
I updated my two Atom devices to 2023.10.3, flipped the wake word switch on both, confirmed they were working, and this morning I literally had to shout directly at one of them to get it out of WAITING_FOR_VAD state. When I did that, it responded correctly. The other Atom is still working fine from across the room. When I toggled the 'Use wake word' switch on the 'deaf' one, I can then start to use it from across the room again.
It seems like there's some progressive/cumulative gain issue that's affecting its ability to detect voice, and therefore not look for the wakeword.
i have the same problem,
Firmware: 2023.12.7 (Jan 17 2024, 01:19:24) Hardware: 1.0
After using it a few times, the display pops up:
Assist in use Off
I then have to reload the configuration.
Same problem here, just got a new M5 Atom Echo. First use after the connection/setup process, wakework works fine, stt and command ok. But the tts seems to be stuck, never ends and I never receive the audio acknowledging the action that was just triggered. And after that, wakeword doesn't work and I need to reset the whole device to get it work, once only.
I'm using the last firmware released (2023.12.9)
Could it be linked with another issue I have. When I try to test the voice I set in the pipeline panel, it throws an error saying: Unable
to load example. Response error: 500.
There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.
I'm personally still experiencing this issue. I'm also following this issue: https://github.com/esphome/issues/issues/5038 which could be a duplicate?
There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.
After installing this yaml, my atom echo has been running for 3 days straight -- no restarts. https://github.com/esphome/firmware/issues/227#issuecomment-2243440334
It uses the newly released micro wake word v2.
that is not the official firmware though, and should not be used as a permanent fix? Jesse wrote this in Discord https://discord.com/channels/429907082951524364/1263109167825158254/1265572325743526009
packages:
m5stack.atom-echo-voice-assistant:
url: https://github.com/esphome/firmware.git
file: voice-assistant/m5stack-atom-echo.adopted.yaml
ref: main
refresh: 0s
and that actually solved the issues.
it still is pointing the firmware to a temporary endpoint I guess, so we will need an update to that at some point.
my complete yaml now is:
substitutions:
name: m5stack-atom-echo-a82d10
friendly_name: M5Stack Atom Echo a82d10
packages:
m5stack.atom-echo-voice-assistant:
url: https://github.com/esphome/firmware.git
file: voice-assistant/m5stack-atom-echo.adopted.yaml
ref: main
refresh: 0s
esphome:
name: ${name}
name_add_mac_suffix: false
friendly_name: ${friendly_name}
api:
encryption:
key: jCb/oWvredacted1RWMmY=
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
sensor:
# Uptime sensor.
- platform: uptime
id: sensor_uptime_echo_library
name: ${name} wifi uptime
- platform: template
id: sensor_free_memory_echo_library
name: ${name} Free Memory
lambda: return heap_caps_get_free_size(MALLOC_CAP_INTERNAL);
unit_of_measurement: B
state_class: measurement
and working just fine
The problem
I've updated my M5 Atom Echo to the latest firmware in order to use Openwakeword which was added with year of voice chapter 4. It works one or two times. Then the device chrashes somehow and loses the connection. Its then no longer possible to wake up the device
What version of Home Assistant Core has the issue?
core-2023.10.2
What was the last working version of Home Assistant Core?
No response
What type of installation are you running?
Home Assistant OS
Integration causing the issue
esphome
Link to integration documentation on our website
https://www.home-assistant.io/integrations/esphome
Diagnostics information
config_entry-esphome-a720417348d184f6b2a67e824c354c52.json.txt
Example YAML snippet
No response
Anything in the logs that might be useful for us?
Additional information
No response