search

lm-sys / FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Apache License 2.0
36.51k stars 4.5k forks source link

ERROR: Exception in ASGI application #2336

Open allenhaozi opened 1 year ago

allenhaozi commented 1 year ago 
2023-08-30 09:03:27 | ERROR | stderr | ERROR:    Exception in ASGI application
2023-08-30 09:03:27 | ERROR | stderr | Traceback (most recent call last):
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/asyncio/selector_events.py", line 848, in _read_ready__data_received
2023-08-30 09:03:27 | ERROR | stderr |     data = self._sock.recv(self.max_size)
2023-08-30 09:03:27 | ERROR | stderr | ConnectionResetError: [Errno 104] Connection reset by peer
2023-08-30 09:03:27 | ERROR | stderr | 
2023-08-30 09:03:27 | ERROR | stderr | The above exception was the direct cause of the following exception:
2023-08-30 09:03:27 | ERROR | stderr | 
2023-08-30 09:03:27 | ERROR | stderr | Traceback (most recent call last):
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
2023-08-30 09:03:27 | ERROR | stderr |     yield
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_backends/anyio.py", line 34, in read
2023-08-30 09:03:27 | ERROR | stderr |     return await self._stream.receive(max_bytes=max_bytes)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 1212, in receive
2023-08-30 09:03:27 | ERROR | stderr |     raise self._protocol.exception
2023-08-30 09:03:27 | ERROR | stderr | anyio.BrokenResourceError
2023-08-30 09:03:27 | ERROR | stderr | 
2023-08-30 09:03:27 | ERROR | stderr | The above exception was the direct cause of the following exception:
2023-08-30 09:03:27 | ERROR | stderr | 
2023-08-30 09:03:27 | ERROR | stderr | Traceback (most recent call last):
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
2023-08-30 09:03:27 | ERROR | stderr |     yield
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
2023-08-30 09:03:27 | ERROR | stderr |     resp = await self._pool.handle_async_request(req)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_async/connection_pool.py", line 262, in handle_async_request
2023-08-30 09:03:27 | ERROR | stderr |     raise exc
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_async/connection_pool.py", line 245, in handle_async_request
2023-08-30 09:03:27 | ERROR | stderr |     response = await connection.handle_async_request(request)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_async/connection.py", line 96, in handle_async_request
2023-08-30 09:03:27 | ERROR | stderr |     return await self._connection.handle_async_request(request)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_async/http11.py", line 121, in handle_async_request
2023-08-30 09:03:27 | ERROR | stderr |     raise exc
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_async/http11.py", line 99, in handle_async_request
2023-08-30 09:03:27 | ERROR | stderr |     ) = await self._receive_response_headers(**kwargs)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_async/http11.py", line 164, in _receive_response_headers
2023-08-30 09:03:27 | ERROR | stderr |     event = await self._receive_event(timeout=timeout)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_async/http11.py", line 200, in _receive_event
2023-08-30 09:03:27 | ERROR | stderr |     data = await self._network_stream.read(
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_backends/anyio.py", line 36, in read
2023-08-30 09:03:27 | ERROR | stderr |     return b""
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/contextlib.py", line 131, in __exit__
2023-08-30 09:03:27 | ERROR | stderr |     self.gen.throw(type, value, traceback)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
2023-08-30 09:03:27 | ERROR | stderr |     raise to_exc(exc) from exc
2023-08-30 09:03:27 | ERROR | stderr | httpcore.ReadError
2023-08-30 09:03:27 | ERROR | stderr | 
2023-08-30 09:03:27 | ERROR | stderr | The above exception was the direct cause of the following exception:
2023-08-30 09:03:27 | ERROR | stderr | 
2023-08-30 09:03:27 | ERROR | stderr | Traceback (most recent call last):
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
2023-08-30 09:03:27 | ERROR | stderr |     result = await app(  # type: ignore[func-returns-value]
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     return await self.app(scope, receive, send)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/fastapi/applications.py", line 289, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     await super().__call__(scope, receive, send)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/applications.py", line 122, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     await self.middleware_stack(scope, receive, send)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     raise exc
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     await self.app(scope, receive, _send)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/middleware/cors.py", line 83, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     await self.app(scope, receive, send)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     raise exc
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     await self.app(scope, receive, sender)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     raise e
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     await self.app(scope, receive, send)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/routing.py", line 718, in __call__
2023-08-30 09:03:27 | ERROR | stderr |     await route.handle(scope, receive, send)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/routing.py", line 276, in handle
2023-08-30 09:03:27 | ERROR | stderr |     await self.app(scope, receive, send)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/starlette/routing.py", line 66, in app
2023-08-30 09:03:27 | ERROR | stderr |     response = await func(request)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/fastapi/routing.py", line 273, in app
2023-08-30 09:03:27 | ERROR | stderr |     raw_response = await run_endpoint_function(
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/fastapi/routing.py", line 190, in run_endpoint_function
2023-08-30 09:03:27 | ERROR | stderr |     return await dependant.call(**values)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/fastchat/serve/openai_api_server.py", line 483, in create_completion
2023-08-30 09:03:27 | ERROR | stderr |     error_check_ret = await check_length(
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/fastchat/serve/openai_api_server.py", line 129, in check_length
2023-08-30 09:03:27 | ERROR | stderr |     response = await client.post(
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_client.py", line 1848, in post
2023-08-30 09:03:27 | ERROR | stderr |     return await self.request(
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_client.py", line 1530, in request
2023-08-30 09:03:27 | ERROR | stderr |     return await self.send(request, auth=auth, follow_redirects=follow_redirects)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_client.py", line 1617, in send
2023-08-30 09:03:27 | ERROR | stderr |     response = await self._send_handling_auth(
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_client.py", line 1645, in _send_handling_auth
2023-08-30 09:03:27 | ERROR | stderr |     response = await self._send_handling_redirects(
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_client.py", line 1682, in _send_handling_redirects
2023-08-30 09:03:27 | ERROR | stderr |     response = await self._send_single_request(request)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_client.py", line 1719, in _send_single_request
2023-08-30 09:03:27 | ERROR | stderr |     response = await transport.handle_async_request(request)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
2023-08-30 09:03:27 | ERROR | stderr |     resp = await self._pool.handle_async_request(req)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/contextlib.py", line 131, in __exit__
2023-08-30 09:03:27 | ERROR | stderr |     self.gen.throw(type, value, traceback)
2023-08-30 09:03:27 | ERROR | stderr |   File "/opt/conda/lib/python3.8/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
2023-08-30 09:03:27 | ERROR | stderr |     raise mapped_exc(message) from exc
2023-08-30 09:03:27 | ERROR | stderr | httpx.ReadError
2023-08-30 09:06:23 | INFO | httpx | HTTP Request: POST http://10.233.69.164:21001/list_models "HTTP/1.1 200 OK"
2023-08-30 09:06:23 | INFO | httpx | HTTP Request: POST http://10.233.69.164:21001/get_worker_address "HTTP/1.1 200 OK"
2023-08-30 09:06:23 | INFO | openai_api_server | model_name: 4pd/qwen-7b-chat@main, worker_addr: http://10.233.69.164:21002
2023-08-30 09:06:23 | INFO | httpx | HTTP Request: POST http://10.233.69.164:21002/model_details "HTTP/1.1 200 OK"
2023-08-30 09:06:23 | INFO | httpx | HTTP Request: POST http://10.233.69.164:21002/count_token "HTTP/1.1 200 OK"
infwinston commented 1 year ago

Hey @allenhaozi could you let us know more details? like what model worker you used and can you reproduce this error?