litgpt download doesn't work

[natanloterio]

Steps to reproduce

• Clone the repository
• Create a new venv environment python -m venv .venv
• Activate venv source .venv/bin/activate
• Run pip install 'litgpt[all]'
• Run litgpt download --repo_id mistralai/Mistral-7B-Instruct-v0.2
• FYI: I was following the Use an LLM for inference section

What I was expecting

• The download of the model mistralai/Mistral-7B-Instruct-v0.2

What I got

(.venv) user@my-machine litgpt % litgpt download --access_token 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 --repo_id mistralai/Mistral-7B-Instruct-v0.2
Traceback (most recent call last):
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status
    response.raise_for_status()
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://huggingface.co/api/models/meta-llama/Llama-2-7b

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/litgpt/scripts/download.py", line 125, in gated_repo_catcher
    yield
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/litgpt/scripts/download.py", line 115, in find_weight_files
    info = repo_info(repo_id, token=access_token)
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 119, in _inner_fn
    return fn(*args, **kwargs)
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/huggingface_hub/hf_api.py", line 2418, in repo_info
    return method(
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 119, in _inner_fn
    return fn(*args, **kwargs)
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/huggingface_hub/hf_api.py", line 2228, in model_info
    hf_raise_for_status(r)
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/huggingface_hub/utils/_errors.py", line 321, in hf_raise_for_status
    raise GatedRepoError(message, response) from e
huggingface_hub.utils._errors.GatedRepoError: 403 Client Error. (Request ID: Root=1-662cdf39-3c42ac8a493c84976da5a8ef;aca97170-45fa-4a3b-b69c-1029281e8627)

Cannot access gated repo for url https://huggingface.co/api/models/meta-llama/Llama-2-7b.
Access to model meta-llama/Llama-2-7b is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-2-7b to ask for access.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/bin/litgpt", line 8, in <module>
    sys.exit(main())
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/litgpt/__main__.py", line 143, in main
    fn(**kwargs)
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/litgpt/scripts/download.py", line 54, in download_from_hub
    bins, safetensors = find_weight_files(repo_id, access_token)
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/litgpt/scripts/download.py", line 115, in find_weight_files
    info = repo_info(repo_id, token=access_token)
  File "/Users/natan.loterio/.pyenv/versions/3.9.1/lib/python3.9/contextlib.py", line 135, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/natan.loterio/workspace/gaia/tmp/litgpt/.venv/lib/python3.9/site-packages/litgpt/scripts/download.py", line 128, in gated_repo_catcher
    raise ValueError(
ValueError: https://huggingface.co/meta-llama/Llama-2-7b requires authentication, please set the `HF_TOKEN=your_token` environment variable or pass --access_token=your_token. You can find your token by visiting https://huggingface.co/settings/tokens

About the error

It seems that the exception is related to a invalid Hugging Face credential, but my credentials are fresh and valid

Suggestion

Improve the error handling so it give a more clear explanation about what must be done in order to get it running properly

[rasbt]

Hi there,

this looks indeed weird. I am not sure why it is showing Llama-2-7b even though you tried to download Mistral.

In my case, when I tried to download it without being authorized, the message is quite clear:

And then with the correct credentials:

I don't have a good explanation why it looks differently for you currently. Is this perhaps for an older LitGPT version? Could you try the latest version?

pip install -U git+https://github.com/Lightning-AI/litgpt.git

[carmocca]

Mistral recently added an auth step to access their models https://huggingface.co/mistralai, so any tutorials that use it need an update to include the access token now

[rasbt]

I can take that. Still weird though that it shows Llama 2 there though. Maybe a side-effect based on a particular access token that was used.

[lantiga]

Instead of a stack trace + error message, we should produce a clean error message (without a stack trace) that provides clear instructions.

[lantiga]

hey @natanloterio we updated the examples so that they don't require a HF token: https://github.com/Lightning-AI/litgpt/pull/1371

the last comment still applies: we should swallow the stack trace and instead just present the message

[rasbt]

We improved the error messages in #1373, thanks again for the feedback!

[natanloterio]

Hello guys, thank you for the support on this issue. I updated my local repo and it worked.

Regarding the divergent logs showing Llama-2-7b and Mistral, it's because I mixed up the copy and paste. Both models were giving me the same error of missing HF_TOKEN.

It seems that a short instruction on how to get an Access Token would be appreciated by the community. So I openend a PR

Let me know what do think.

Cheers