search

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://docs.vllm.ai
Apache License 2.0
26.72k stars 3.91k forks source link

[Bug]: vllm.engine.async_llm_engine.AsyncEngineDeadError: Task finished unexpectedly. #7987

Open TangJiakai opened 2 weeks ago

TangJiakai commented 2 weeks ago

Your current environment

The error output ```text Exception in ASGI application Traceback (most recent call last): File "/data/tangjiakai/anaconda3/lib/python3.11/site-packages/anyio/streams/memory.py", line 81, in receive return self.receive_nowait() ^^^^^^^^^^^^^^^^^^^^^ File "/data/tangjiakai/anaconda3/lib/python3.11/site-packages/anyio/streams/memory.py", line 76, in receive_nowait raise WouldBlock anyio.WouldBlock During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/data/tangjiakai/anaconda3/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 419, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/tangjiakai/anaconda3/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__ return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/tangjiakai/anaconda3/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "/data/tangjiakai/anaconda3/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__ await self.middleware_stack(scope, receive, send) File "/data/tangjiakai/anaconda3/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__ await self.app(scope, receive, _send) File "/data/tangjiakai/anaconda3/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__ response = await self.dispatch_func(request, call_next) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/tangjiakai/anaconda3/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 361, in authentication return await call_next(request) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/tangjiakai/anaconda3/lib/python3.11/site-packages/starlette/middleware/base.py", line 159, in call_next message = await recv_stream.receive() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/tangjiakai/anaconda3/lib/python3.11/site-packages/anyio/streams/memory.py", line 89, in receive await receive_event.wait() File "/data/tangjiakai/anaconda3/lib/python3.11/asyncio/locks.py", line 213, in wait await fut asyncio.exceptions.CancelledError ```

🐛 Describe the bug

I have tried:

  1. --enforce-eager
  2. --disable-frontend-multiprocessing
  3. --disable-custom-all-reduce
  4. --distributed-executor-backend mp

however, no one is working..

Before submitting a new issue...

TangJiakai commented 2 weeks ago

AsyncLLMEngine is already dead, terminating server proces

BaiMoHan commented 2 weeks ago

same, https://github.com/vllm-project/vllm/issues/7936