Open paducz opened 4 weeks ago
Same issue reported here https://github.com/home-assistant/core/issues/108666
Found same issue reported in custom Elevenlabs TTS integration. The root cause is the same https://github.com/carleeno/elevenlabs_tts/issues/129
Hi @paducz
custom_components/elevenlabs_tts
is a custom component, which is not maintained by the HA core project. Please report your issue to the maintainer or in the source repository of this custom component.
Thx 👍
Hi @paducz
custom_components/elevenlabs_tts
is a custom component, which is not maintained by the HA core project. Please report your issue to the maintainer or in the source repository of this custom component. Thx 👍
Hi this is not related to the custom component. This is happening also with core integration Elevenlabs and another integrations for TTS - see my comments above...from my perspective this problem is in core TTS
Here same log when using core integration Elevenlabs, in https://github.com/home-assistant/core/issues/108666 is reported the same when when using Legacy component..so this seems to be a problem in the core TTS module
2024-08-12 11:06:13.314 DEBUG (MainThread) [homeassistant.components.elevenlabs.tts] Options: {'voice': 'XX3Kz776E1c9WxVIwVkk'}
2024-08-12 11:06:18.653 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback SpeechManager._async_get_tts_audio.<locals>.handle_error(<Task cancell...nit__.py:745>>) at /usr/src/homeassistant/homeassistant/components/tts/__init__.py:811 (None)
Traceback (most recent call last):
File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 813, in handle_error
if audio_task.exception():
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 755, in get_tts_data
extension, data = await engine_instance.internal_async_get_tts_audio(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 489, in internal_async_get_tts_audio
return await self.async_get_tts_audio(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/elevenlabs/tts.py", line 110, in async_get_tts_audio
bytes_combined = b"".join([byte_seg async for byte_seg in audio])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/elevenlabs/text_to_speech/client.py", line 648, in convert
async with self._client_wrapper.httpx_client.stream(
File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/elevenlabs/core/http_client.py", line 440, in stream
async with self.httpx_client.stream(
File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1617, in stream
response = await self.send(
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1661, in send
response = await self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1689, in _send_handling_auth
response = await self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1726, in _send_handling_redirects
response = await self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1763, in _send_single_request
response = await transport.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 373, in handle_async_request
resp = await self._pool.handle_async_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request
raise exc from None
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request
response = await connection.handle_async_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection.py", line 101, in handle_async_request
return await self._connection.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 143, in handle_async_request
raise exc
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 113, in handle_async_request
) = await self._receive_response_headers(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 186, in _receive_response_headers
event = await self._receive_event(timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 224, in _receive_event
data = await self._network_stream.read(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpcore/_backends/anyio.py", line 35, in read
return await self._stream.receive(max_bytes=max_bytes)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/anyio/streams/tls.py", line 205, in receive
data = await self._call_sslobject_method(self._ssl_object.read, max_bytes)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/anyio/streams/tls.py", line 147, in _call_sslobject_method
data = await self.transport_stream.receive()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 1142, in receive
await self._protocol.read_event.wait()
File "/usr/local/lib/python3.12/asyncio/locks.py", line 212, in wait
await fut
asyncio.exceptions.CancelledError
Hey there @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (tts
) you are listed as a code owner for? Thanks!
(message by CodeOwnersMention)
tts documentation tts source (message by IssueLinks)
Hey there @sorgfresser, mind taking a look at this issue as it has been labeled with an integration (elevenlabs
) you are listed as a code owner for? Thanks!
(message by CodeOwnersMention)
elevenlabs documentation elevenlabs source (message by IssueLinks)
Just a note - when using model Eleven Turbo v2.5 in elevenlabs, all is good..this happens only with Eleven Multilingual v2, which probably costs more time to generate the voice
the timeout is caused by the httpx
client which is instantiated in the elevenlabs
library, so i don't think it's an issue in the tts
component itself.
Agreed that this is an elevenlabs issue. We can fix this by setting timeout accordingly (https://github.com/elevenlabs/elevenlabs-python/blob/44e805e8f9c3d50ec951166b173d7c47a4143e4d/src/elevenlabs/client.py#L55)
But I'm not sure I want to increase this variable too much, e.g. timeout 0. The setting of this timeout value is something I would like to discuss with someone more knowledgeable in terms of home assistant.
Thank you guys for taking care of it. What I am curious is that in the code you are reffering to is timeout 60s (at least based on the comment), but generating the voice takes just a few seconds for 800 chars. Or am I mixing things?
Nope, you're not mixing things. That's why I asked for a configuration.yaml which can reproduce the problem (which it sadly can't for me, the yaml you shared works on my machine). Still, I would like to know if an increase in the timeout would solve this problem for you. But still, the voice creation is rather fast for me as well, so I am a bit confused on what exactly is going on, but sadly can't reproduce it yet. I can give you a version with increased timeout and you can install it as a custom integration and we could see whether this fixes it. Would that work for you?
Sure I can try modified version as a custom integration. Just to be sure - are you using model Eleven Multilingual v2? Model Eleven Turbo v2.5 works fine dor me as well
Good point, both models work fine for me. Please, find a modified version in my fork (https://github.com/sorgfresser/homeassistant-core/tree/elevenlabs-httpx-timeout) I would be glad if you could verify whether this fixes it
Thank you for sharing this, but I am not a experienced user of HA, and I am not sure I loaded it as as custom integration correctly....What I did: I copied all the files from https://github.com/sorgfresser/homeassistant-core/tree/elevenlabs-httpx-timeout/homeassistant/components/elevenlabs to config/custom_components/elevenlabs, but I do not think HA loaded it - Am I doing that right?
Sorry for the late response. So yeah, that sounds about right. The reason why it's not loading should be the missing version key in the manifest.json. If you replace your manifest.json with this
{
"domain": "elevenlabs",
"version": "0.0.1",
"name": "ElevenLabs",
"codeowners": ["@sorgfresser"],
"config_flow": true,
"documentation": "https://www.home-assistant.io/integrations/elevenlabs",
"integration_type": "service",
"iot_class": "cloud_polling",
"loggers": ["elevenlabs"],
"requirements": ["elevenlabs==1.6.1"]
}
It should load
Thank you, now it loaded now...unfortunately the error persists
Error doing job: Exception in callback SpeechManager._async_get_tts_audio.<locals>.handle_error(<Task cancell...nit__.py:746>>) at /usr/src/homeassistant/homeassistant/components/tts/__init__.py:812 (None)
Traceback (most recent call last):
File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 814, in handle_error
if audio_task.exception():
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 756, in get_tts_data
extension, data = await engine_instance.internal_async_get_tts_audio(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 490, in internal_async_get_tts_audio
return await self.async_get_tts_audio(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/elevenlabs/tts.py", line 110, in async_get_tts_audio
bytes_combined = b"".join([byte_seg async for byte_seg in audio])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/elevenlabs/text_to_speech/client.py", line 648, in convert
async with self._client_wrapper.httpx_client.stream(
File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/elevenlabs/core/http_client.py", line 440, in stream
async with self.httpx_client.stream(
File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1617, in stream
response = await self.send(
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1661, in send
response = await self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1689, in _send_handling_auth
response = await self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1726, in _send_handling_redirects
response = await self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1763, in _send_single_request
response = await transport.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 373, in handle_async_request
resp = await self._pool.handle_async_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request
raise exc from None
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request
response = await connection.handle_async_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection.py", line 101, in handle_async_request
return await self._connection.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 143, in handle_async_request
raise exc
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 113, in handle_async_request
) = await self._receive_response_headers(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 186, in _receive_response_headers
event = await self._receive_event(timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 224, in _receive_event
data = await self._network_stream.read(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/httpcore/_backends/anyio.py", line 35, in read
return await self._stream.receive(max_bytes=max_bytes)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/anyio/streams/tls.py", line 205, in receive
data = await self._call_sslobject_method(self._ssl_object.read, max_bytes)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/anyio/streams/tls.py", line 147, in _call_sslobject_method
data = await self.transport_stream.receive()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 1142, in receive
await self._protocol.read_event.wait()
File "/usr/local/lib/python3.12/asyncio/locks.py", line 212, in wait
await fut
asyncio.exceptions.CancelledError
The problem
The new Elevenlabs integration offers high quality models such as eleven_multilingual_v2. Problem with this model is that it takes some time to generate longer messages, which probably leads to timeout in the TTS module:
What version of Home Assistant Core has the issue?
2024.8.1
What was the last working version of Home Assistant Core?
No response
What type of installation are you running?
Home Assistant OS
Integration causing the issue
No response
Link to integration documentation on our website
No response
Diagnostics information
No response
Example YAML snippet
No response
Anything in the logs that might be useful for us?
No response
Additional information
No response