scripts/run-13b.sh fails with http 401 from huggingface.co URLs

[DrPaulBrewer]

Environment: Google Cloud, Nvidia A100 40 GB, 12vCPU, 100 GB disk Docker and CUDA 12.1 are installed.

This part is OK:

git clone https://github.com/soulteary/docker-llama2-chat scripts/make-13b.sh

Access from Google VM to huggingface.co seems to be ok (ping 10-12ms)

This part FAILS.

scripts/run-13b.sh

Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 261, in hf_raise_for_status response.raise_for_status() File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-13b-chat-hf/resolve/main/config.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 417, in cached_file resolved_file = hf_hub_download( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, *kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1195, in hf_hub_download metadata = get_hf_file_metadata( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1541, in get_hf_file_metadata hf_raise_for_status(r) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 293, in hf_raise_for_status raise RepositoryNotFoundError(message, response) from e huggingface_hub.utils._errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-64c0a70b-7e218e8a7f87e86a5fbfb030;382d0c02-cba0-4312-b459-953c3d6951bb)

Repository Not Found for url: https://huggingface.co/meta-llama/Llama-2-13b-chat-hf/resolve/main/config.json. Please make sure you specified the correct repo_id and repo_type. If you are trying to access a private or gated repo, make sure you are authenticated. Invalid username or password.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/app/app.py", line 6, in from model import run File "/app/model.py", line 10, in config = AutoConfig.from_pretrained(model_id) File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 983, in from_pretrained config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, kwargs) File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 617, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, kwargs) File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 672, in _get_config_dict resolved_config_file = cached_file( File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 433, in cached_file raise EnvironmentError( OSError: meta-llama/Llama-2-13b-chat-hf is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models' If this is a private repository, make sure to pass a token having permission to this repo with use_auth_token or log in with huggingface-cli login and pass use_auth_token=True.

[DrPaulBrewer]

We did NOT run your step #2 before running your step 3 "scripts/run-13b.sh"

Step 2 in your docs includes downloading a model that is supplied into the docker container by run script, particularly one of:

git clone https://huggingface.co/meta-llama/Llama-2-7b-chat-hf git clone https://huggingface.co/meta-llama/Llama-2-13b-chat-hf

Either of these commands presents a username/password prompt. Perhaps they are not public access?

[soulteary]

hey, thank you for your feedback. i'll provide a one-click tools.