litgpt download doesn't work

[natanloterio]

Steps to reproduce

• Clone the repository
• Create a new venv environment python -m venv .venv
• Activate venv source .venv/bin/activate
• Run pip install 'litgpt[all]'
• Run litgpt download --repo_id mistralai/Mistral-7B-Instruct-v0.2
• FYI: I was following the Use an LLM for inference section

What I was expecting

• The download of the model mistralai/Mistral-7B-Instruct-v0.2

What I got

(.venv) user@my-machine litgpt % litgpt download --access_token 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 --repo_id mistralai/Mistral-7B-Instruct-v0.2
Traceback (most recent call last):
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status
    response.raise_for_status()
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://huggingface.co/api/models/meta-llama/Llama-2-7b

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/litgpt/scripts/download.py", line 125, in gated_repo_catcher
    yield
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/litgpt/scripts/download.py", line 115, in find_weight_files
    info = repo_info(repo_id, token=access_token)
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 119, in _inner_fn
    return fn(*args, **kwargs)
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/huggingface_hub/hf_api.py", line 2418, in repo_info
    return method(
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 119, in _inner_fn
    return fn(*args, **kwargs)
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/huggingface_hub/hf_api.py", line 2228, in model_info
    hf_raise_for_status(r)
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/huggingface_hub/utils/_errors.py", line 321, in hf_raise_for_status
    raise GatedRepoError(message, response) from e
huggingface_hub.utils._errors.GatedRepoError: 403 Client Error. (Request ID: Root=1-662cdf39-3c42ac8a493c84976da5a8ef;aca97170-45fa-4a3b-b69c-1029281e8627)

Cannot access gated repo for url https://huggingface.co/api/models/meta-llama/Llama-2-7b.
Access to model meta-llama/Llama-2-7b is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-2-7b to ask for access.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/bin/litgpt", line 8, in <module>
    sys.exit(main())
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/litgpt/__main__.py", line 143, in main
    fn(**kwargs)
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/litgpt/scripts/download.py", line 54, in download_from_hub
    bins, safetensors = find_weight_files(repo_id, access_token)
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/litgpt/scripts/download.py", line 115, in find_weight_files
    info = repo_info(repo_id, token=access_token)
  File "/Users/natan.loterio/.pyenv/versions/3.9.1/lib/python3.9/contextlib.py", line 135, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/litgpt/scripts/download.py", line 128, in gated_repo_catcher
    raise ValueError(
ValueError: https://huggingface.co/meta-llama/Llama-2-7b requires authentication, please set the `HF_TOKEN=your_token` environment variable or pass --access_token=your_token. You can find your token by visiting https://huggingface.co/settings/tokens

About the error

It seems that the exception is related to a invalid Hugging Face credential, but my credentials are fresh and valid

Suggestion

Improve the error handling so it give a more clear explanation about what must be done in order to get it running properly

[rasbt]

Hi there,

this looks indeed weird. I am not sure why it is showing Llama-2-7b even though you tried to download Mistral.

In my case, when I tried to download it without being authorized, the message is quite clear:

And then with the correct credentials:

I don't have a good explanation why it looks differently for you currently. Is this perhaps for an older LitGPT version? Could you try the latest version?

pip install -U git+https://github.com/Lightning-AI/litgpt.git

[carmocca]

Mistral recently added an auth step to access their models https://huggingface.co/mistralai, so any tutorials that use it need an update to include the access token now

[rasbt]

I can take that. Still weird though that it shows Llama 2 there though. Maybe a side-effect based on a particular access token that was used.

[lantiga]

Instead of a stack trace + error message, we should produce a clean error message (without a stack trace) that provides clear instructions.

[lantiga]

hey @natanloterio we updated the examples so that they don't require a HF token: https://github.com/Lightning-AI/litgpt/pull/1371

the last comment still applies: we should swallow the stack trace and instead just present the message

[rasbt]

We improved the error messages in #1373, thanks again for the feedback!

[natanloterio]

Hello guys, thank you for the support on this issue. I updated my local repo and it worked.

Regarding the divergent logs showing Llama-2-7b and Mistral, it's because I mixed up the copy and paste. Both models were giving me the same error of missing HF_TOKEN.

It seems that a short instruction on how to get an Access Token would be appreciated by the community. So I openend a PR

Let me know what do think.

Cheers

[Bennyggcc]

I create the access token, and run the command ''litgpt download meta-llama/Meta-Llama-3-8B-Instruct --access_token='', but it still report error. How should I address it?

The terminal output as the following: Setting HF_HUB_ENABLE_HF_TRANSFER=1 Traceback (most recent call last): File "/home/ubuntu/anaconda3/envs/Bin_LLM/bin/litgpt", line 8, in sys.exit(main()) ^^^^^^ File "/home/ubuntu/anaconda3/envs/Bin_LLM/lib/python3.12/site-packages/litgpt/main.py", line 71, in main CLI(parser_data) File "/home/ubuntu/anaconda3/envs/Bin_LLM/lib/python3.12/site-packages/jsonargparse/_cli.py", line 119, in CLI return _run_component(component, init.get(subcommand)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/Bin_LLM/lib/python3.12/site-packages/jsonargparse/_cli.py", line 204, in _run_component return component(**cfg) ^^^^^^^^^^^^^^^^ File "/home/ubuntu/anaconda3/envs/Bin_LLM/lib/python3.12/site-packages/litgpt/scripts/download.py", line 85, in download_from_hub with gated_repo_catcher(repo_id, access_token): File "/home/ubuntu/anaconda3/envs/Bin_LLM/lib/python3.12/contextlib.py", line 158, in exit self.gen.throw(value) File "/home/ubuntu/anaconda3/envs/Bin_LLM/lib/python3.12/site-packages/litgpt/scripts/download.py", line 158, in gated_repo_catcher raise ValueError( ValueError: https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct requires authentication. The access token provided by HF_TOKEN=your_token environment variable or --access_token=your_token may not have sufficient access rights. Please visit https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct for more information.

[rasbt]

Hi there,

the specific models also require that you accept the license terms on the individual model pages. For the model you are interested in, have you accepted these terms? E.g., if you go to https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct, there should be a form on the landing page. After accepting the terms, it should look like this:

[Bennyggcc]

Hi there,

the specific models also require that you accept the license terms on the individual model pages. For the model you are interested in, have you accepted these terms? E.g., if you go to https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct, there should be a form on the landing page. After accepting the terms, it should look like this:

OK, I get it. I have not been granted access to this model. I try to get the access right. Thank you!