dosubot[bot]
commented
6 months ago The error you're encountering is due to asyncio tasks being attached to a different event loop than the one currently running. This is a common issue when there's a mismatch between the event loop used by FastAPI/Uvicorn and other asyncio tasks or background tasks in your application. Here are steps to address this:
- Ensure all asynchronous operations are correctly awaited and compatible with FastAPI's event loop.
- If your application creates new event loops, use
asyncio.set_event_loop(new_loop)to set the newly created loop as the current event loop before scheduling tasks on it. - Review FastAPI's background tasks to ensure they don't inadvertently create a new event loop or run in a different loop.
- Increase Uvicorn/Gunicorn workers to distribute the load, especially since you're aiming for 500 queries per minute. Use the
--workersargument to specify the number of worker processes. - Implement connection pooling for database access and ensure your database client library is async-compatible.
- Optimize your async code to avoid blocking operations and ensure non-blocking behavior.
- Implement rate limiting to manage the load and prevent overloading your application.
- Perform load testing to identify and address any concurrency issues or bottlenecks.
For more detailed guidance, refer to the FastAPI documentation and the Uvicorn documentation. This approach should help you resolve the issue and scale your application to handle the desired load.
Sources
- [BUG] RuntimeError: Task <Task pending name='Task-1329' coro=<RequestResponseCycle.run_asgi() running at /data/soft ware/anaconda3/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py:419> cb=[set.discard()]> got Future
attached to a different loop - server/api.py
- requirements_api.txt
SwordHG
ShAw7ock
Nancy7zt
版本是0.29版本 调用的是在线的qwen-turbo 基于知识的问答，当并发大于5的时候，报错 ERROR: Exception in ASGI application Traceback (most recent call last): File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 419, in run_asgi result = await app( # type: ignore[func-returns-value] File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in call return await self.app(scope, receive, send) File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/fastapi/applications.py", line 1054, in call await super().call(scope, receive, send) File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/starlette/applications.py", line 116, in call await self.middleware_stack(scope, receive, send) File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/starlette/middleware/errors.py", line 186, in call raise exc File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/starlette/middleware/errors.py", line 164, in call await self.app(scope, receive, _send) File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 62, in call await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/starlette/_exception_handler.py", line 55, in wrapped_app raise exc File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/starlette/_exception_handler.py", line 44, in wrapped_app await app(scope, receive, sender) File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/starlette/routing.py", line 746, in call await route.handle(scope, receive, send) File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/starlette/routing.py", line 288, in handle await self.app(scope, receive, send) File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/starlette/routing.py", line 75, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/starlette/_exception_handler.py", line 55, in wrapped_app raise exc File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/starlette/_exception_handler.py", line 44, in wrapped_app await app(scope, receive, sender) File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/starlette/routing.py", line 70, in app response = await func(request) File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/fastapi/routing.py", line 299, in app raise e File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/fastapi/routing.py", line 294, in app raw_response = await run_endpoint_function( File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/fastapi/routing.py", line 191, in run_endpoint_function return await dependant.call(**values) File "/data/zhouyj/IntQA/env/lib/python3.9/site-packages/fastchat/serve/base_model_worker.py", line 196, in api_generate_stream await acquire_worker_semaphore() File "/usr/local/python3.9/lib/python3.9/asyncio/locks.py", line 417, in acquire await fut RuntimeError: Task <Task pending name='Task-820' coro=<RequestResponseCycle.run_asgi() running at /data/zhouyj/IntQA/env/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py:419> cb=[set.discard()]> got Future attached to a different loop
2024-04-12 14:56:25 | INFO | httpx | HTTP Request: POST http://127.0.0.1:21006/worker_generate_stream "HTTP/1.1 500 Internal Server Error"
这个问题怎么解决，现在qwen-turbo支持qpm为500，想要支持qpm500的并发