Closed maitagorri closed 1 month ago
cc @ArthurZucker
We figured out that the download with a write-enabled token does work, but not with a read-enabled one. This might be a bug with huggingface's authentication, rather than AutoTokenizer?
@maitagorri Huh, interesting, thanks for sharing this info. Maybe cc @Wauplin here, the hub master :)
We figured out that the download with a write-enabled token does work, but not with a read-enabled one
That's unexpected indeed. @coyotte508 any idea?
Hm @maitagorri do you have a request id - or the token name+user? Maybe it's a fine-grained token and not a read token that has the issue?
If it's a fine-grained token - we just deployed a new checkbox to allow access to all gated repos - which should fix your issue:
Otherwise I would be interested in the user+token name so we can investigate (or request id header of the response)
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
System Info
Using
transformers
version: 4.41.2Who can help?
No response
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
I am attempting to create a Tokenizer via download from a gated model site on huggingface,
mistralai/Mixtral-8x7B-Instruct-v0.1
, using a token for authentication:This results in an OSError that looks as if I couldn't reach huggingface at all (expand for traceback):
OSError referencing generic inability to read model
``` --------------------------------------------------------------------------- HTTPError Traceback (most recent call last) File c:\Users\schadem\Anaconda3\envs\watermark-env\Lib\site-packages\huggingface_hub\utils\_errors.py:304, in hf_raise_for_status(response, endpoint_name) [303](file:///C:/Users/schadem/Anaconda3/envs/watermark-env/Lib/site-packages/huggingface_hub/utils/_errors.py:303) try: --> [304](file:///C:/Users/schadem/Anaconda3/envs/watermark-env/Lib/site-packages/huggingface_hub/utils/_errors.py:304) response.raise_for_status() [305](file:///C:/Users/schadem/Anaconda3/envs/watermark-env/Lib/site-packages/huggingface_hub/utils/_errors.py:305) except HTTPError as e: File c:\Users\schadem\Anaconda3\envs\watermark-env\Lib\site-packages\requests\models.py:1024, in Response.raise_for_status(self) [1023](file:///C:/Users/schadem/Anaconda3/envs/watermark-env/Lib/site-packages/requests/models.py:1023) if http_error_msg: -> [1024](file:///C:/Users/schadem/Anaconda3/envs/watermark-env/Lib/site-packages/requests/models.py:1024) raise HTTPError(http_error_msg, response=self) HTTPError: 403 Client Error: Forbidden for url: https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1/resolve/main/config.json The above exception was the direct cause of the following exception: HfHubHTTPError Traceback (most recent call last) File c:\Users\schadem\Anaconda3\envs\watermark-env\Lib\site-packages\huggingface_hub\file_download.py:1722, in _get_metadata_or_catch_error(repo_id, filename, repo_type, revision, endpoint, proxies, etag_timeout, headers, local_files_only, relative_filename, storage_folder) [1721](file:///C:/Users/schadem/Anaconda3/envs/watermark-env/Lib/site-packages/huggingface_hub/file_download.py:1721) try: -> [1722](file:///C:/Users/schadem/Anaconda3/envs/watermark-env/Lib/site-packages/huggingface_hub/file_download.py:1722) metadata = get_hf_file_metadata(url=url, proxies=proxies, timeout=etag_timeout, headers=headers) [1723](file:///C:/Users/schadem/Anaconda3/envs/watermark-env/Lib/site-packages/huggingface_hub/file_download.py:1723) except EntryNotFoundError as http_error: File c:\Users\schadem\Anaconda3\envs\watermark-env\Lib\site-packages\huggingface_hub\utils\_validators.py:114, in validate_hf_hub_args.Expected behavior
I would expect the same result as with download of an open model, which works just fine, e.g.
I am able to access the model through the website when logged in.
Attempting the download with an invalid token, I get the expected error (expand for traceback):
OSError referencing gated model