search

WisdomShell / codeshell-vscode

An intelligent coding assistant plugin for Visual Studio Code, developed based on CodeShell
Apache License 2.0
569 stars 69 forks source link

通过text-generation-inference部署时报错 #35

Open Halflifefa opened 8 months ago

Halflifefa commented 8 months ago

命令如下

docker run --gpus 'all' --shm-size 1g -p 9090:80 -v $HOME/codeshell/CodeShell-7B-Chat:/data \
        --env LOG_LEVEL="info,text_generation_router=debug" \
        ghcr.nju.edu.cn/huggingface/text-generation-inference:1.0.3 \
        --model-id WisdomShell/CodeShell-7B-Chat-int4 --num-shard 1 \
        --max-total-tokens 5000 --max-input-length 4096 \
        --max-stop-sequences 12 --trust-remote-code

2023-10-24T07:02:11.814270Z INFO download: text_generation_launcher: Starting download process. Error: DownloadError 2023-10-24T07:02:16.019924Z ERROR download: text_generation_launcher: Download encountered an error: Traceback (most recent call last):

File "/opt/conda/lib/python3.9/site-packages/urllib3/connection.py", line 203, in _new_conn sock = connection.create_connection(

File "/opt/conda/lib/python3.9/site-packages/urllib3/util/connection.py", line 85, in create_connection raise err

File "/opt/conda/lib/python3.9/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa)

TimeoutError: [Errno 110] Connection timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "/opt/conda/lib/python3.9/site-packages/urllib3/connectionpool.py", line 790, in urlopen response = self._make_request(

File "/opt/conda/lib/python3.9/site-packages/urllib3/connectionpool.py", line 491, in _make_request raise new_e

File "/opt/conda/lib/python3.9/site-packages/urllib3/connectionpool.py", line 467, in _make_request self._validate_conn(conn)

File "/opt/conda/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1092, in _validate_conn conn.connect()

File "/opt/conda/lib/python3.9/site-packages/urllib3/connection.py", line 611, in connect self.sock = sock = self._new_conn()

File "/opt/conda/lib/python3.9/site-packages/urllib3/connection.py", line 218, in _new_conn raise NewConnectionError(

urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f1b325b6fd0>: Failed to establish a new connection: [Errno 110] Connection timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "/opt/conda/lib/python3.9/site-packages/requests/adapters.py", line 486, in send resp = conn.urlopen(

File "/opt/conda/lib/python3.9/site-packages/urllib3/connectionpool.py", line 844, in urlopen retries = retries.increment(

File "/opt/conda/lib/python3.9/site-packages/urllib3/util/retry.py", line 515, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]

urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/CodeShell-7B-Chat (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f1b325b6fd0>: Failed to establish a new connection: [Errno 110] Connection timed out'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/opt/conda/bin/text-generation-server", line 8, in sys.exit(app())

File "/opt/conda/lib/python3.9/site-packages/text_generation_server/cli.py", line 113, in download_weights utils.weight_files(model_id, revision, extension)

File "/opt/conda/lib/python3.9/site-packages/text_generation_server/utils/hub.py", line 96, in weight_files filenames = weight_hub_files(model_id, revision, extension)

File "/opt/conda/lib/python3.9/site-packages/text_generation_server/utils/hub.py", line 25, in weight_hub_files info = api.model_info(model_id, revision=revision)

File "/opt/conda/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, **kwargs)

File "/opt/conda/lib/python3.9/site-packages/huggingface_hub/hf_api.py", line 1677, in model_info r = get_session().get(path, headers=headers, timeout=timeout, params=params)

File "/opt/conda/lib/python3.9/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs)

File "/opt/conda/lib/python3.9/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs)

File "/opt/conda/lib/python3.9/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs)

File "/opt/conda/lib/python3.9/site-packages/huggingface_hub/utils/_http.py", line 63, in send return super().send(request, *args, **kwargs)

File "/opt/conda/lib/python3.9/site-packages/requests/adapters.py", line 519, in send raise ConnectionError(e, request=request)

requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/CodeShell-7B-Chat (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f1b325b6fd0>: Failed to establish a new connection: [Errno 110] Connection timed out'))"), '(Request ID: a7ec4c93-df44-453f-95e5-7e027bc442b9)')

国内的网络环境连接不上huggingface

DongchenZhou commented 7 months ago

俺也一样