[BUG] Async (nested sync detected) error in UI when summarizing large/long text files

Stego72 commented 2 months ago

Pre-check

[X] I have searched the existing issues and none cover this bug.

Description

Using Docker (Ollama-CUDA default settings)

A "RuntimeError: Detected nested async" is raised when trying to summarize large documents from the UI. The model is generating the output in the trace but it fail to appear in the UI - after a while the RuntimeError is raised.

It appears to be a consequence of a previous runtime error:

File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request raise exc from None File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 189, in handle_async_request await self._close_connections(closing) File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 305, in _close_connections await connection.aclose() File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_async/connection.py", line 171, in aclose await self._connection.aclose() File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_async/http11.py", line 265, in aclose await self._network_stream.aclose() File "/home/worker/app/.venv/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 55, in aclose await self._stream.aclose() File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 1198, in aclose self._transport.write_eof() File "uvloop/handles/stream.pyx", line 699, in uvloop.loop.UVStream.write_eof File "uvloop/handles/handle.pyx", line 159, in uvloop.loop.UVHandle._ensure_alive RuntimeError: unable to perform operation on <TCPTransport closed=True reading=False 0x7fd41cdb1090>; the handler is closed

Steps to Reproduce

Ingest large PDF document (150 pages or more)
From the UI, prompt to summarize document
Observe response being generated in the trace
RuntimeError is raised after a while

Expected Behavior

Summary provided

Actual Behavior

RuntimeError

Environment

Win10, RTX4090, Docker install (vanilla Ollama-GPU) as per installation guide

Additional Information

No response

Version

No response

Setup Checklist

[X] Confirm that you have followed the installation instructions in the project’s documentation.
[X] Check that you are using the latest version of the project.
[X] Verify disk space availability for model storage and data processing.
[X] Ensure that you have the necessary permissions to run the project.

NVIDIA GPU Setup Checklist

[X] Check that the all CUDA dependencies are installed and are compatible with your GPU (refer to CUDA's documentation)
[X] Ensure an NVIDIA GPU is installed and recognized by the system (run nvidia-smi to verify).
[X] Ensure proper permissions are set for accessing GPU resources.
[X] Docker users - Verify that the NVIDIA Container Toolkit is configured correctly (e.g. run sudo docker run --rm --gpus all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi)