search

tetele / onju-voice-satellite

An ESPHome config for the Onju Voice which makes it a Home Assistant voice satellite
MIT License
105 stars 18 forks source link

Does not play audio, only the answers #13

Open fantnhu opened 8 months ago

fantnhu commented 8 months ago

Hi! I have 4 of these devices, all working as Voice Assistant. However, I can't play media (mp3, wav) on any of them. I have tried many times, but it only reads the reply messages. As a media player, it's like it doesn't even work. I tried to send everything over to it with services, TTS, but I couldn't even click on the media player entity in ESPHome integration and select a file to play audio with it.

Any ideas? Thanks: István Pintér

PhilippeP62 commented 8 months ago

First disable Wake Word Detection, use media_player , re-enable wwd.

fantnhu commented 8 months ago

First disable Wake Word Detection, use media_player , re-enable wwd.

Thank you. Of course I started with this. I've tried it a million times. I've been developing for more than 20 years, I think I've tried everything there is to try

fantnhu commented 8 months ago

It took me a lot of work before I could describe the problem well. So I have tried 3 ways to play sound, 2 of which don't work.

1: Basic - Doesn't work, the way HomeAssistant sends the audio, using its own domain Cloudflare. https://MYDOMAIN.HU/local/audio/event_start.wav (No HA error log entry)

Device log:

[07:41:28][D][media_player:059]: 'None' - Setting
[07:41:28][D][media_player:066]:   Media URL: https://MYDOMAIN/local/audio/event_start.wav?authSig=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiI5MmI4OGE0OWRmNGE0ZjM0YWU5NThlZTE0YjIyYTllNiIsInBhdGgiOiIvbG9jYWwvYXVkaW8vZXZlbnRfc3RhcnQud2F2IiwicGFyYW1zIjpbXSwiaWF0IjoxNzExMDg5Njg4LCJleHAiOjE3MTExNzYwODh9.d0Bbwq1xUrEf0oYWX9lcep_vXJdIXF4rxj56L4GuEwo
[07:41:29][W][component:232]: Component i2s_audio.media_player took a long time for an operation (521 ms).
[07:41:29][W][component:233]: Components should block for at most 30 ms.
[07:41:29][D][media_player:059]: 'None' - Setting
[07:41:29][D][media_player:066]:   Media URL: https://MYDOMAIN/local/audio/event_start.wav?authSig=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiI5MmI4OGE0OWRmNGE0ZjM0YWU5NThlZTE0YjIyYTllNiIsInBhdGgiOiIvbG9jYWwvYXVkaW8vZXZlbnRfc3RhcnQud2F2IiwicGFyYW1zIjpbXSwiaWF0IjoxNzExMDg5Njg4LCJleHAiOjE3MTExNzYwODh9.d0Bbwq1xUrEf0oYWX9lcep_vXJdIXF4rxj56L4GuEwo
[07:41:30][W][component:232]: Component api took a long time for an operation (510 ms).
[07:41:30][W][component:233]: Components should block for at most 30 ms.
[07:41:30][W][component:232]: Component i2s_audio.media_player took a long time for an operation (507 ms).
[07:41:30][W][component:233]: Components should block for at most 30 ms.

2: mDNS based - Doesn't work and the device reboots every time. https://homeassistant.local:8123/local/audio/event_start.wav

HA log entry:

Napló: aioesphomeapi.connection
Forrás: runner.py:188
Először fordult elő: 07:44:48 (1 Események)
Utoljára naplózott: 07:44:48

onju-voice-dfbacc @ 192.168.31.156: Connection error occurred: [Errno 104] Connection reset by peer
Napló: homeassistant.components.cast.media_player
Forrás: components/cast/media_player.py:379
integráció: Google Cast (dokumentáció, problémák)
Először fordult elő: 07:44:25 (2 Események)
Utoljára naplózott: 07:44:25

Failed to cast media https://homeassistant.local:8123/local/audio/event_start.wav. Please make sure the URL is: Reachable from the cast device and either a publicly resolvable hostname or an IP address

The URL works, I can play the audio from my browser

Device log:

[07:44:25][D][media_player:059]: 'None' - Setting
[07:44:25][D][media_player:066]:   Media URL: https://homeassistant.local:8123/local/audio/event_start.wav

3: Based on HA IP - Works, plays the audio (only with https url, http is not correct). https://192.168.31.10:8123/local/audio/event_start.wav

Device log:

[07:51:03][D][media_player:059]: 'None' - Setting
[07:51:03][D][media_player:066]:   Media URL: https://192.168.31.10:8123/local/audio/event_start.wav
[07:51:05][W][component:232]: Component i2s_audio.media_player took a long time for an operation (1439 ms).
[07:51:05][W][component:233]: Components should block for at most 30 ms.
[07:51:07][W][component:232]: Component i2s_audio.media_player took a long time for an operation (61 ms).
[07:51:07][W][component:233]: Components should block for at most 30 ms.
[07:51:07][W][component:232]: Component i2s_audio.media_player took a long time for an operation (185 ms).
[07:51:07][W][component:233]: Components should block for at most 30 ms.
tetele commented 8 months ago

If the IP-based URL works, this sounds more like an issue with your networking than the actual firmware.

However, let me check. Please paste your full config and specify the versions of HA and ESPHome you are using.

ther3zz commented 7 months ago

Did you ever get this figured out?

I'm running into a similar issue except mine wont play any audio at all...

[16:40:08][D][voice_assistant:416]: State changed from STOPPING_MICROPHONE to AWAITING_RESPONSE
[16:40:11][D][voice_assistant:523]: Event Type: 4
[16:40:11][D][voice_assistant:551]: Speech recognised as: " What time is it?"
[16:40:11][D][voice_assistant:523]: Event Type: 5
[16:40:11][D][voice_assistant:556]: Intent started
[16:40:11][D][voice_assistant:523]: Event Type: 6
[16:40:11][D][voice_assistant:523]: Event Type: 7
[16:40:11][D][voice_assistant:579]: Response: "Sorry, I couldn't understand that"
[16:40:11][D][voice_assistant:523]: Event Type: 8
[16:40:11][D][voice_assistant:599]: Response URL: "https://homeassistant.my.domain/api/tts_proxy/dae2cdcb27a1d1c3b07ba2c7db91480f9d4bfd8f_en-us_7238ee98e6_marytts.mp3"
[16:40:11][D][voice_assistant:416]: State changed from AWAITING_RESPONSE to STREAMING_RESPONSE
[16:40:11][D][voice_assistant:422]: Desired state set to STREAMING_RESPONSE
[16:40:11][D][media_player:059]: 'Office Onju 2a44d8' - Setting
[16:40:11][D][media_player:066]:   Media URL: https://homeassistant.my.domain/api/tts_proxy/dae2cdcb27a1d1c3b07ba2c7db91480f9d4bfd8f_en-us_7238ee98e6_marytts.mp3
[16:40:11][D][media_player:059]: 'Office Onju 2a44d8' - Setting
[16:40:11][D][media_player:066]:   Media URL: https://homeassistant.my.domain/api/tts_proxy/dae2cdcb27a1d1c3b07ba2c7db91480f9d4bfd8f_en-us_7238ee98e6_marytts.mp3
[16:40:11][D][light:036]: 'top_led' Setting:
[16:40:11][D][light:059]:   Red: 20%, Green: 100%, Blue: 0%
[16:40:11][D][light:109]:   Effect: 'speaking'
[16:40:11][D][voice_assistant:523]: Event Type: 2
[16:40:11][D][voice_assistant:613]: Assist Pipeline ended
[16:40:11][D][light:036]: 'top_led' Setting:
[16:40:11][D][light:109]:   Effect: 'show_volume'
[16:40:11][W][component:232]: Component i2s_audio.media_player took a long time for an operation (525 ms).
[16:40:11][W][component:233]: Components should block for at most 30 ms.
[16:40:12][W][component:232]: Component i2s_audio.media_player took a long time for an operation (501 ms).
[16:40:12][W][component:233]: Components should block for at most 30 ms.
[16:40:12][D][light:036]: 'top_led' Setting:
[16:40:12][D][light:051]:   Brightness: 60%
[16:40:12][D][light:059]:   Red: 100%, Green: 0%, Blue: 100%
[16:40:12][D][light:109]:   Effect: 'listening_ww'
[16:40:12][D][light:036]: 'top_led' Setting:
[16:40:12][D][light:051]:   Brightness: 60%
[16:40:12][D][light:059]:   Red: 100%, Green: 0%, Blue: 100%
[16:40:12][D][light:085]:   Transition length: 0.1s

I confirmed that the URLs for the mp3s are accessible via other devices on the same network... I'm also not really having issues with my esp32 s3 box

tetele commented 7 months ago

It's not complaining about not being able to access the URLs, so that's not the issue.

Also, it's not the same issue as the original one, as OP complained about streaming all other audio apart from voice responses, whereas you say yours doesn't even play those. I'd suggest opening up a different issue.

tetele commented 7 months ago

34 sounds like exactly the same issue. Check out https://github.com/tetele/onju-voice-satellite/issues/34#issuecomment-2061703927

ther3zz commented 7 months ago

Hi! I have 4 of these devices, all working as Voice Assistant. However, I can't play media (mp3, wav) on any of them. I have tried many times, but it only reads the reply messages. As a media player, it's like it doesn't even work. I tried to send everything over to it with services, TTS, but I couldn't even click on the media player entity in ESPHome integration and select a file to play audio with it.

Any ideas? Thanks: István Pintér

So I just figured this out. I noticed that while speaking to the assistant I would get responses but if I attempted to playback an MP3 or TTS via media it would not work. You have to turn off the Wake Word switch and then you can playback stuff.

If you have an automation that playback specific TTS or mp3s, you can set an action to turn off the wake word switch ->delay 1 seond -> playback tts/mp3 -> delay for the length of the audio played -> turn on the switch

tetele commented 7 months ago

You have to turn off the Wake Word switch and then you can playback stuff.

That was already suggested and tried