chongchongaikubao
commented
1 month ago 我又尝试了一下，这是完整报错： (xinference) zhuofei@zhuofei-System-Product-Name:~$ XINFERENCE_DISABLE_HEALTH_CHECK=1 xinference-local --host 0.0.0.0 --port 9998 2024-07-19 18:21:11,667 xinference.core.supervisor 418444 INFO Xinference supervisor 0.0.0.0:56406 started 2024-07-19 18:21:11,715 xinference.core.worker 418444 INFO Starting metrics export server at 0.0.0.0:None 2024-07-19 18:21:11,716 xinference.core.worker 418444 INFO Checking metrics export server... 2024-07-19 18:21:12,981 xinference.core.worker 418444 INFO Metrics server is started at: http://0.0.0.0:45423 2024-07-19 18:21:12,981 xinference.core.worker 418444 INFO Xinference worker 0.0.0.0:56406 started 2024-07-19 18:21:12,981 xinference.core.worker 418444 INFO Purge cache directory: /home/zhuofei/.xinference/cache 2024-07-19 18:21:17,042 xinference.api.restful_api 418332 INFO Starting Xinference at endpoint: http://0.0.0.0:9998 2024-07-19 18:21:17,176 uvicorn.error 418332 INFO Uvicorn running on http://0.0.0.0:9998 (Press CTRL+C to quit) 2024-07-19 18:23:45,715 xinference.model.llm.llm_family 418444 INFO Caching from Hugging Face: Qwen/Qwen-1_8B-Chat 2024-07-19 18:25:56,913 xinference.model.utils 418444 WARNING Attempt 1 failed. Remaining attempts: 2 2024-07-19 18:28:07,985 xinference.model.utils 418444 WARNING Attempt 2 failed. Remaining attempts: 1 2024-07-19 18:28:08,020 xinference.model.utils 418444 WARNING Attempt 3 failed. Remaining attempts: 0 2024-07-19 18:28:08,020 xinference.core.worker 418444 ERROR Failed to load model qwen-chat-1-0 Traceback (most recent call last): File "/home/zhuofei/.local/lib/python3.11/site-packages/urllib3/connection.py", line 174, in _new_conn conn = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/urllib3/util/connection.py", line 95, in create_connection raise err File "/home/zhuofei/.local/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection sock.connect(sa) OSError: [Errno 101] Network is unreachable
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/home/zhuofei/.local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 715, in urlopen httplib_response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 404, in _make_request self._validate_conn(conn) File "/home/zhuofei/.local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1058, in _validate_conn conn.connect() File "/home/zhuofei/.local/lib/python3.11/site-packages/urllib3/connection.py", line 363, in connect self.sock = conn = self._new_conn() ^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/urllib3/connection.py", line 186, in _new_conn raise NewConnectionError( urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7fdf8ad0dc90>: Failed to establish a new connection: [Errno 101] Network is unreachable
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/home/zhuofei/.local/lib/python3.11/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 799, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/urllib3/util/retry.py", line 592, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/Qwen/Qwen-1_8B-Chat/revision/c3db8007171847931da7efa4b2ed4309afcce021 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fdf8ad0dc90>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/home/zhuofei/.local/lib/python3.11/site-packages/huggingface_hub/_snapshot_download.py", line 164, in snapshot_download repo_info = api.repo_info(repo_id=repo_id, repo_type=repo_type, revision=revision, token=token) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(*args, kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2491, in repo_info return method( ^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(args, kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 2300, in model_info r = get_session().get(path, headers=headers, timeout=timeout, params=params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 66, in send return super().send(request, args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/Qwen/Qwen-1_8B-Chat/revision/c3db8007171847931da7efa4b2ed4309afcce021 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fdf8ad0dc90>: Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: a52e2470-f302-458e-978d-00d54e62a265)')
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/model/utils.py", line 104, in retry_download return download_func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/huggingface_hub/_snapshot_download.py", line 236, in snapshot_download raise LocalEntryNotFoundError( huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the files on the Hub and we cannot find the appropriate snapshot folder for the specified revision on the local disk. Please check your internet connection and try again.
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/core/worker.py", line 816, in launch_builtin_model model, model_description = await asyncio.to_thread( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/asyncio/threads.py", line 25, in to_thread return await loop.run_in_executor(None, func_call) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/model/core.py", line 69, in create_model_instance return create_llm_model_instance( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/model/llm/core.py", line 224, in create_llm_model_instance save_path = cache(llm_family, llm_spec, quantization) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/model/llm/llm_family.py", line 321, in cache return cache_from_huggingface(llm_family, llm_spec, quantization) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/model/llm/llm_family.py", line 853, in cache_from_huggingface download_dir = retry_download( ^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/model/utils.py", line 127, in retry_download raise RuntimeError( RuntimeError: Failed to download model 'qwen-chat' (size: 1_8, format: pytorch) after multiple retries 2024-07-19 18:28:08,036 xinference.api.restful_api 418332 ERROR [address=0.0.0.0:56406, pid=418444] Failed to download model 'qwen-chat' (size: 1_8, format: pytorch) after multiple retries huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the files on the Hub and we cannot find the appropriate snapshot folder for the specified revision on the local disk. Please check your internet connection and try again.
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/api/restful_api.py", line 835, in launch_model model_uid = await (await self._get_supervisor_ref()).launch_builtin_model( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/xoscar/backends/context.py", line 231, in send return self._process_result_message(result) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/xoscar/backends/context.py", line 102, in _process_result_message raise message.as_instanceof_cause() File "/home/zhuofei/.local/lib/python3.11/site-packages/xoscar/backends/pool.py", line 656, in send result = await self._run_coro(message.message_id, coro) ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/xoscar/backends/pool.py", line 367, in _run_coro return await coro File "/home/zhuofei/.local/lib/python3.11/site-packages/xoscar/api.py", line 384, in on_receive return await super().on_receive(message) # type: ignore ^^^^^^^^^^^^^^^^^ File "xoscar/core.pyx", line 558, in on_receive__ raise ex File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.on_receive async with self._lock: ^^^^^^^^^^^^^^^^^ File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.on_receive with debug_async_timeout('actor_lock_timeout', ^^^^^^^^^^^^^^^^^ File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive result = await result ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/core/supervisor.py", line 988, in launch_builtin_model await _launch_model() ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/core/supervisor.py", line 952, in _launch_model await _launch_one_model(rep_model_uid) ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/core/supervisor.py", line 932, in _launch_one_model await worker_ref.launch_builtin_model( ^^^^^^^^^^^^^^^^^ File "xoscar/core.pyx", line 284, in __pyx_actor_method_wrapper async with lock: File "xoscar/core.pyx", line 287, in xoscar.core.__pyx_actor_method_wrapper result = await result ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/core/utils.py", line 45, in wrapped ret = await func(*args, *kwargs) ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/core/worker.py", line 816, in launch_builtin_model model, model_description = await asyncio.to_thread( ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/asyncio/threads.py", line 25, in to_thread return await loop.run_in_executor(None, func_call) ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/model/core.py", line 69, in create_model_instance return create_llm_model_instance( ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/model/llm/core.py", line 224, in create_llm_model_instance save_path = cache(llm_family, llm_spec, quantization) ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/model/llm/llm_family.py", line 321, in cache return cache_from_huggingface(llm_family, llm_spec, quantization) ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/model/llm/llm_family.py", line 853, in cache_from_huggingface download_dir = retry_download( ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.conda/envs/xinference/lib/python3.11/site-packages/xinference/model/utils.py", line 127, in retry_download raise RuntimeError( ^^^^^^^^^^^^^^^^^ RuntimeError: [address=0.0.0.0:56406, pid=418444] Failed to download model 'qwen-chat' (size: 1_8, format: pytorch) after multiple retries
qinxuye
System Info / 系統信息
cuda12.2 linux
Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？
Version info / 版本信息
xinference 0.13.1
The command used to start Xinference / 用以启动 xinference 的命令
xinference-local -H 0.0.0.0
Reproduction / 复现过程
新建python3.11环境，安装xinference,运行xinference-local -H 0.0.0.0 (xinference) zhuofei@zhuofei:~$ xinference-local -H 0.0.0.0 2024-07-19 17:44:08,412 xinference.core.supervisor 364834 INFO Xinference supervisor 0.0.0.0:19124 started 2024-07-19 17:44:08,444 xinference.core.worker 364834 INFO Starting metrics export server at 0.0.0.0:None 2024-07-19 17:44:08,445 xinference.core.worker 364834 INFO Checking metrics export server... 2024-07-19 17:44:10,618 xinference.core.worker 364834 INFO Metrics server is started at: http://0.0.0.0:35019 2024-07-19 17:44:10,618 xinference.core.worker 364834 INFO Xinference worker 0.0.0.0:19124 started 2024-07-19 17:44:10,618 xinference.core.worker 364834 INFO Purge cache directory: /home/zhuofei/.xinference/cache 2024-07-19 17:44:10,618 xinference.core.utils 364834 INFO Remove empty directory: /home/zhuo/.xinference/cache/qwen1.5-chat-pytorch-0_5b 2024-07-19 17:44:13,776 xinference.api.restful_api 364725 INFO Starting Xinference at endpoint: http://0.0.0.0:9997 2024-07-19 17:44:13,895 xinference.api.restful_api 364725 WARNING Failed to create socket with port 9997 2024-07-19 17:44:13,914 xinference.api.restful_api 364725 INFO Found available port: 39617 2024-07-19 17:44:13,914 xinference.api.restful_api 364725 INFO Starting Xinference at endpoint: http://0.0.0.0:39617 2024-07-19 17:44:13,945 uvicorn.error 364725 INFO Uvicorn running on http://0.0.0.0:39617 (Press CTRL+C to quit) Traceback (most recent call last): File "/home/zhuofei/.local/lib/python3.11/site-packages/xinference/api/restful_api.py", line 835, in launch_model model_uid = await (await self._get_supervisor_ref()).launch_builtin_model( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/xoscar/backends/context.py", line 231, in send return self._process_result_message(result) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/xoscar/backends/context.py", line 102, in _process_result_message raise message.as_instanceof_cause() File "/home/zhuofei/.local/lib/python3.11/site-packages/xoscar/backends/pool.py", line 656, in send result = await self._run_coro(message.message_id, coro) ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/xoscar/backends/pool.py", line 367, in _run_coro return await coro File "/home/zhuofei/.local/lib/python3.11/site-packages/xoscar/api.py", line 384, in on_receive return await super().on_receive(message) # type: ignore ^^^^^^^^^^^^^^^^^ File "xoscar/core.pyx", line 558, in on_receive__ raise ex File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.on_receive async with self._lock: ^^^^^^^^^^^^^^^^^ File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.on_receive with debug_async_timeout('actor_lock_timeout', ^^^^^^^^^^^^^^^^^ File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive result = await result ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/xinference/core/supervisor.py", line 988, in launch_builtin_model await _launch_model() ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/xinference/core/supervisor.py", line 952, in _launch_model await _launch_one_model(rep_model_uid) ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/xinference/core/supervisor.py", line 932, in _launch_one_model await worker_ref.launch_builtin_model( ^^^^^^^^^^^^^^^^^ File "xoscar/core.pyx", line 284, in __pyx_actor_method_wrapper async with lock: File "xoscar/core.pyx", line 287, in xoscar.core.__pyx_actor_method_wrapper result = await result ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/xinference/core/utils.py", line 45, in wrapped ret = await func(*args, *kwargs) ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/xinference/core/worker.py", line 816, in launch_builtin_model model, model_description = await asyncio.to_thread( ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/workspace/anaconda3/lib/python3.11/asyncio/threads.py", line 25, in to_thread return await loop.run_in_executor(None, func_call) ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/workspace/anaconda3/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/xinference/model/core.py", line 69, in create_model_instance return create_llm_model_instance( ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/xinference/model/llm/core.py", line 224, in create_llm_model_instance save_path = cache(llm_family, llm_spec, quantization) ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/xinference/model/llm/llm_family.py", line 321, in cache return cache_from_huggingface(llm_family, llm_spec, quantization) ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/xinference/model/llm/llm_family.py", line 853, in cache_from_huggingface download_dir = retry_download( ^^^^^^^^^^^^^^^^^ File "/home/zhuofei/.local/lib/python3.11/site-packages/xinference/model/utils.py", line 127, in retry_download raise RuntimeError( ^^^^^^^^^^^^^^^^^ RuntimeError: [address=192.168.100.125:19276, pid=172070] Failed to download model 'qwen-chat' (size: 1_8, format: pytorch) after multiple retries
Expected behavior / 期待表现
模型无法下载