home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
71.11k stars 29.79k forks source link

Error when generating longer messages via TTS module #123845

Open paducz opened 4 weeks ago

paducz commented 4 weeks ago

The problem

The new Elevenlabs integration offers high quality models such as eleven_multilingual_v2. Problem with this model is that it takes some time to generate longer messages, which probably leads to timeout in the TTS module:

2024-08-13 22:42:37.654 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback SpeechManager._async_get_tts_audio.<locals>.handle_error(<Task cancell...nit__.py:745>>) at /usr/src/homeassistant/homeassistant/components/tts/__init__.py:811 (None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 813, in handle_error
    if audio_task.exception():
       ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 755, in get_tts_data
    extension, data = await engine_instance.internal_async_get_tts_audio(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 489, in internal_async_get_tts_audio
    return await self.async_get_tts_audio(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/elevenlabs_tts/tts.py", line 121, in async_get_tts_audio
    return await self._client.get_tts_audio(message, options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/elevenlabs_tts/elevenlabs.py", line 155, in get_tts_audio
    resp = await self.post(endpoint, data, params, api_key=api_key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/elevenlabs_tts/elevenlabs.py", line 84, in post
    response = await self.session.post(
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1892, in post
    return await self.request(
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1574, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1661, in send
    response = await self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1689, in _send_handling_auth
    response = await self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1726, in _send_handling_redirects
    response = await self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1763, in _send_single_request
    response = await transport.handle_async_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 373, in handle_async_request
    resp = await self._pool.handle_async_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request
    raise exc from None
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request
    response = await connection.handle_async_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection.py", line 101, in handle_async_request
    return await self._connection.handle_async_request(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 143, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 113, in handle_async_request
    ) = await self._receive_response_headers(**kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 186, in _receive_response_headers
    event = await self._receive_event(timeout=timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 224, in _receive_event
    data = await self._network_stream.read(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_backends/anyio.py", line 35, in read
    return await self._stream.receive(max_bytes=max_bytes)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/streams/tls.py", line 205, in receive
    data = await self._call_sslobject_method(self._ssl_object.read, max_bytes)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/streams/tls.py", line 147, in _call_sslobject_method
    data = await self.transport_stream.receive()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 1142, in receive
    await self._protocol.read_event.wait()
  File "/usr/local/lib/python3.12/asyncio/locks.py", line 212, in wait
    await fut
asyncio.exceptions.CancelledError

What version of Home Assistant Core has the issue?

2024.8.1

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

No response

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

paducz commented 4 weeks ago

Same issue reported here https://github.com/home-assistant/core/issues/108666

paducz commented 4 weeks ago

Found same issue reported in custom Elevenlabs TTS integration. The root cause is the same https://github.com/carleeno/elevenlabs_tts/issues/129

mib1185 commented 4 weeks ago

Hi @paducz custom_components/elevenlabs_tts is a custom component, which is not maintained by the HA core project. Please report your issue to the maintainer or in the source repository of this custom component. Thx 👍

paducz commented 4 weeks ago

Hi @paducz custom_components/elevenlabs_tts is a custom component, which is not maintained by the HA core project. Please report your issue to the maintainer or in the source repository of this custom component. Thx 👍

Hi this is not related to the custom component. This is happening also with core integration Elevenlabs and another integrations for TTS - see my comments above...from my perspective this problem is in core TTS

paducz commented 4 weeks ago

Here same log when using core integration Elevenlabs, in https://github.com/home-assistant/core/issues/108666 is reported the same when when using Legacy component..so this seems to be a problem in the core TTS module

2024-08-12 11:06:13.314 DEBUG (MainThread) [homeassistant.components.elevenlabs.tts] Options: {'voice': 'XX3Kz776E1c9WxVIwVkk'}
2024-08-12 11:06:18.653 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback SpeechManager._async_get_tts_audio.<locals>.handle_error(<Task cancell...nit__.py:745>>) at /usr/src/homeassistant/homeassistant/components/tts/__init__.py:811 (None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 813, in handle_error
    if audio_task.exception():
       ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 755, in get_tts_data
    extension, data = await engine_instance.internal_async_get_tts_audio(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 489, in internal_async_get_tts_audio
    return await self.async_get_tts_audio(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/elevenlabs/tts.py", line 110, in async_get_tts_audio
    bytes_combined = b"".join([byte_seg async for byte_seg in audio])
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/elevenlabs/text_to_speech/client.py", line 648, in convert
    async with self._client_wrapper.httpx_client.stream(
  File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/elevenlabs/core/http_client.py", line 440, in stream
    async with self.httpx_client.stream(
  File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1617, in stream
    response = await self.send(
               ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1661, in send
    response = await self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1689, in _send_handling_auth
    response = await self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1726, in _send_handling_redirects
    response = await self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1763, in _send_single_request
    response = await transport.handle_async_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 373, in handle_async_request
    resp = await self._pool.handle_async_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request
    raise exc from None
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request
    response = await connection.handle_async_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection.py", line 101, in handle_async_request
    return await self._connection.handle_async_request(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 143, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 113, in handle_async_request
    ) = await self._receive_response_headers(**kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 186, in _receive_response_headers
    event = await self._receive_event(timeout=timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 224, in _receive_event
    data = await self._network_stream.read(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_backends/anyio.py", line 35, in read
    return await self._stream.receive(max_bytes=max_bytes)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/streams/tls.py", line 205, in receive
    data = await self._call_sslobject_method(self._ssl_object.read, max_bytes)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/streams/tls.py", line 147, in _call_sslobject_method
    data = await self.transport_stream.receive()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 1142, in receive
    await self._protocol.read_event.wait()
  File "/usr/local/lib/python3.12/asyncio/locks.py", line 212, in wait
    await fut
asyncio.exceptions.CancelledError
home-assistant[bot] commented 4 weeks ago

Hey there @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (tts) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `tts` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign tts` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

(message by CodeOwnersMention)


tts documentation tts source (message by IssueLinks)

home-assistant[bot] commented 4 weeks ago

Hey there @sorgfresser, mind taking a look at this issue as it has been labeled with an integration (elevenlabs) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `elevenlabs` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign elevenlabs` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

(message by CodeOwnersMention)


elevenlabs documentation elevenlabs source (message by IssueLinks)

paducz commented 4 weeks ago

Just a note - when using model Eleven Turbo v2.5 in elevenlabs, all is good..this happens only with Eleven Multilingual v2, which probably costs more time to generate the voice

mib1185 commented 4 weeks ago

the timeout is caused by the httpx client which is instantiated in the elevenlabs library, so i don't think it's an issue in the tts component itself.

sorgfresser commented 3 weeks ago

Agreed that this is an elevenlabs issue. We can fix this by setting timeout accordingly (https://github.com/elevenlabs/elevenlabs-python/blob/44e805e8f9c3d50ec951166b173d7c47a4143e4d/src/elevenlabs/client.py#L55)

But I'm not sure I want to increase this variable too much, e.g. timeout 0. The setting of this timeout value is something I would like to discuss with someone more knowledgeable in terms of home assistant.

paducz commented 3 weeks ago

Thank you guys for taking care of it. What I am curious is that in the code you are reffering to is timeout 60s (at least based on the comment), but generating the voice takes just a few seconds for 800 chars. Or am I mixing things?

sorgfresser commented 3 weeks ago

Nope, you're not mixing things. That's why I asked for a configuration.yaml which can reproduce the problem (which it sadly can't for me, the yaml you shared works on my machine). Still, I would like to know if an increase in the timeout would solve this problem for you. But still, the voice creation is rather fast for me as well, so I am a bit confused on what exactly is going on, but sadly can't reproduce it yet. I can give you a version with increased timeout and you can install it as a custom integration and we could see whether this fixes it. Would that work for you?

paducz commented 3 weeks ago

Sure I can try modified version as a custom integration. Just to be sure - are you using model Eleven Multilingual v2? Model Eleven Turbo v2.5 works fine dor me as well

sorgfresser commented 3 weeks ago

Good point, both models work fine for me. Please, find a modified version in my fork (https://github.com/sorgfresser/homeassistant-core/tree/elevenlabs-httpx-timeout) I would be glad if you could verify whether this fixes it

paducz commented 3 weeks ago

Thank you for sharing this, but I am not a experienced user of HA, and I am not sure I loaded it as as custom integration correctly....What I did: I copied all the files from https://github.com/sorgfresser/homeassistant-core/tree/elevenlabs-httpx-timeout/homeassistant/components/elevenlabs to config/custom_components/elevenlabs, but I do not think HA loaded it - Am I doing that right?

sorgfresser commented 2 days ago

Sorry for the late response. So yeah, that sounds about right. The reason why it's not loading should be the missing version key in the manifest.json. If you replace your manifest.json with this

{
  "domain": "elevenlabs",
  "version": "0.0.1",
  "name": "ElevenLabs",
  "codeowners": ["@sorgfresser"],
  "config_flow": true,
  "documentation": "https://www.home-assistant.io/integrations/elevenlabs",
  "integration_type": "service",
  "iot_class": "cloud_polling",
  "loggers": ["elevenlabs"],
  "requirements": ["elevenlabs==1.6.1"]
}

It should load

paducz commented 2 days ago

Thank you, now it loaded now...unfortunately the error persists

Error doing job: Exception in callback SpeechManager._async_get_tts_audio.<locals>.handle_error(<Task cancell...nit__.py:746>>) at /usr/src/homeassistant/homeassistant/components/tts/__init__.py:812 (None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 814, in handle_error
    if audio_task.exception():
       ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 756, in get_tts_data
    extension, data = await engine_instance.internal_async_get_tts_audio(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 490, in internal_async_get_tts_audio
    return await self.async_get_tts_audio(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/elevenlabs/tts.py", line 110, in async_get_tts_audio
    bytes_combined = b"".join([byte_seg async for byte_seg in audio])
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/elevenlabs/text_to_speech/client.py", line 648, in convert
    async with self._client_wrapper.httpx_client.stream(
  File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/elevenlabs/core/http_client.py", line 440, in stream
    async with self.httpx_client.stream(
  File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1617, in stream
    response = await self.send(
               ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1661, in send
    response = await self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1689, in _send_handling_auth
    response = await self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1726, in _send_handling_redirects
    response = await self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1763, in _send_single_request
    response = await transport.handle_async_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 373, in handle_async_request
    resp = await self._pool.handle_async_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request
    raise exc from None
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request
    response = await connection.handle_async_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection.py", line 101, in handle_async_request
    return await self._connection.handle_async_request(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 143, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 113, in handle_async_request
    ) = await self._receive_response_headers(**kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 186, in _receive_response_headers
    event = await self._receive_event(timeout=timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 224, in _receive_event
    data = await self._network_stream.read(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_backends/anyio.py", line 35, in read
    return await self._stream.receive(max_bytes=max_bytes)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/streams/tls.py", line 205, in receive
    data = await self._call_sslobject_method(self._ssl_object.read, max_bytes)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/streams/tls.py", line 147, in _call_sslobject_method
    data = await self.transport_stream.receive()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 1142, in receive
    await self._protocol.read_event.wait()
  File "/usr/local/lib/python3.12/asyncio/locks.py", line 212, in wait
    await fut
asyncio.exceptions.CancelledError