[X] I have searched the existing issues and none cover this bug.
Description
Using Docker (Ollama-CUDA default settings)
A "RuntimeError: Detected nested async" is raised when trying to summarize large documents from the UI. The model is generating the output in the trace but it fail to appear in the UI - after a while the RuntimeError is raised.
It appears to be a consequence of a previous runtime error:
File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request
raise exc from None
File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 189, in handle_async_request
await self._close_connections(closing)
File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 305, in _close_connections
await connection.aclose()
File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_async/connection.py", line 171, in aclose
await self._connection.aclose()
File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_async/http11.py", line 265, in aclose
await self._network_stream.aclose()
File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 55, in aclose
await self._stream.aclose()
File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 1198, in aclose
self._transport.write_eof()
File "uvloop/handles/stream.pyx", line 699, in uvloop.loop.UVStream.write_eof
File "uvloop/handles/handle.pyx", line 159, in uvloop.loop.UVHandle._ensure_alive
RuntimeError: unable to perform operation on <TCPTransport closed=True reading=False 0x7fd41cdb1090>; the handler is closed
Steps to Reproduce
Ingest large PDF document (150 pages or more)
From the UI, prompt to summarize document
Observe response being generated in the trace
RuntimeError is raised after a while
Expected Behavior
Summary provided
Actual Behavior
RuntimeError
Environment
Win10, RTX4090, Docker install (vanilla Ollama-GPU) as per installation guide
Additional Information
No response
Version
No response
Setup Checklist
[X] Confirm that you have followed the installation instructions in the project’s documentation.
[X] Check that you are using the latest version of the project.
[X] Verify disk space availability for model storage and data processing.
[X] Ensure that you have the necessary permissions to run the project.
NVIDIA GPU Setup Checklist
[X] Check that the all CUDA dependencies are installed and are compatible with your GPU (refer to CUDA's documentation)
[X] Ensure an NVIDIA GPU is installed and recognized by the system (run nvidia-smi to verify).
[X] Ensure proper permissions are set for accessing GPU resources.
[X] Docker users - Verify that the NVIDIA Container Toolkit is configured correctly (e.g. run sudo docker run --rm --gpus all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi)
Pre-check
Description
Using Docker (Ollama-CUDA default settings)
A "RuntimeError: Detected nested async" is raised when trying to summarize large documents from the UI. The model is generating the output in the trace but it fail to appear in the UI - after a while the RuntimeError is raised.
It appears to be a consequence of a previous runtime error:
File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request raise exc from None File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 189, in handle_async_request await self._close_connections(closing) File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 305, in _close_connections await connection.aclose() File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_async/connection.py", line 171, in aclose await self._connection.aclose() File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_async/http11.py", line 265, in aclose await self._network_stream.aclose() File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 55, in aclose await self._stream.aclose() File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 1198, in aclose self._transport.write_eof() File "uvloop/handles/stream.pyx", line 699, in uvloop.loop.UVStream.write_eof File "uvloop/handles/handle.pyx", line 159, in uvloop.loop.UVHandle._ensure_alive RuntimeError: unable to perform operation on <TCPTransport closed=True reading=False 0x7fd41cdb1090>; the handler is closed
Steps to Reproduce
Expected Behavior
Summary provided
Actual Behavior
RuntimeError
Environment
Win10, RTX4090, Docker install (vanilla Ollama-GPU) as per installation guide
Additional Information
No response
Version
No response
Setup Checklist
NVIDIA GPU Setup Checklist
nvidia-smi
to verify).sudo docker run --rm --gpus all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi
)