[bug] `token` not supported in `AutoModel`

[ain-soph]

I see use_auth_token is already deprecated and will be replaced by token, https://github.com/huggingface/transformers/blob/b257c46a075419c09e5ce5c5aa39bc346ecdb9a5/src/transformers/modeling_utils.py#L2196-L2204

But AutoModel doesn't accept the new argument token yet. Especially in https://github.com/huggingface/transformers/blob/b257c46a075419c09e5ce5c5aa39bc346ecdb9a5/src/transformers/configuration_utils.py#L631-L638

Directly call LlamaForCausalLM instead of AutoModelForCausalLM is a temporary workaround to get rid of the deprecation warning.

Reproduction

transformers.AutoModelForCausalLM.from_pretrained('meta-llama/Llama-2-7b-chat-hf',use_auth_token='XXX')
transformers.LlamaForCausalLM.from_pretrained('meta-llama/Llama-2-7b-chat-hf',token='XXX')

Expected behavior

AutoModel should support new token argument.

[sgugger]

Could you let us know which version of transformers you are using? I just tried this on the main branch and it seems to work fine.

[ain-soph]

@sgugger

$ pip freeze | grep transformers
transformers @ git+https://github.com/huggingface/transformers.git@b08f41e62a41632195cb986fcc41d428a5bf1d56

Error Log for token

>>> transformers.AutoModelForCausalLM.from_pretrained('meta-llama/Llama-2-7b-chat-hf', token='■■■■■■')
Traceback (most recent call last):
  File "/home/renpang/miniconda3/envs/py311/lib/python3.11/site-packages/huggingface_hub/utils/_errors.py", line 259, in hf_raise_for_status
    response.raise_for_status()
  File "/home/renpang/miniconda3/envs/py311/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/renpang/miniconda3/envs/py311/lib/python3.11/site-packages/transformers/utils/hub.py", line 417, in cached_file
    resolved_file = hf_hub_download(
                    ^^^^^^^^^^^^^^^^
  File "/home/renpang/miniconda3/envs/py311/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/renpang/miniconda3/envs/py311/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1195, in hf_hub_download
    metadata = get_hf_file_metadata(
               ^^^^^^^^^^^^^^^^^^^^^
  File "/home/renpang/miniconda3/envs/py311/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/renpang/miniconda3/envs/py311/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1541, in get_hf_file_metadata
    hf_raise_for_status(r)
  File "/home/renpang/miniconda3/envs/py311/lib/python3.11/site-packages/huggingface_hub/utils/_errors.py", line 291, in hf_raise_for_status
    raise RepositoryNotFoundError(message, response) from e
huggingface_hub.utils._errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=■■■■■■)

Repository Not Found for url: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json.
Please make sure you specified the correct `repo_id` and `repo_type`.
If you are trying to access a private or gated repo, make sure you are authenticated.
Invalid username or password.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/renpang/miniconda3/envs/py311/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 461, in from_pretrained
    config, kwargs = AutoConfig.from_pretrained(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/renpang/miniconda3/envs/py311/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 983, in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/renpang/miniconda3/envs/py311/lib/python3.11/site-packages/transformers/configuration_utils.py", line 617, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/renpang/miniconda3/envs/py311/lib/python3.11/site-packages/transformers/configuration_utils.py", line 672, in _get_config_dict
    resolved_config_file = cached_file(
                           ^^^^^^^^^^^^
  File "/home/renpang/miniconda3/envs/py311/lib/python3.11/site-packages/transformers/utils/hub.py", line 433, in cached_file
    raise EnvironmentError(
OSError: meta-llama/Llama-2-7b-chat-hf is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo with `use_auth_token` or log in with `huggingface-cli login` and pass `use_auth_token=True`.

Deprecation Log for use_auth_token

>>> transformers.AutoModelForCausalLM.from_pretrained('meta-llama/Llama-2-7b-chat-hf',use_auth_token='■■■■■■')
[2023-07-24 15:51:53,994] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect)
/home/renpang/miniconda3/envs/py311/lib/python3.11/site-packages/transformers/modeling_utils.py:2197: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers.
  warnings.warn(
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:01<00:00,  1.04it/s]
LlamaForCausalLM(
  (model): LlamaModel(
    (embed_tokens): Embedding(32000, 4096, padding_idx=0)
    (layers): ModuleList(
      (0-31): 32 x LlamaDecoderLayer(
        (self_attn): LlamaAttention(
          (q_proj): Linear(in_features=4096, out_features=4096, bias=False)
          (k_proj): Linear(in_features=4096, out_features=4096, bias=False)
          (v_proj): Linear(in_features=4096, out_features=4096, bias=False)
          (o_proj): Linear(in_features=4096, out_features=4096, bias=False)
          (rotary_emb): LlamaRotaryEmbedding()
        )
        (mlp): LlamaMLP(
          (gate_proj): Linear(in_features=4096, out_features=11008, bias=False)
          (up_proj): Linear(in_features=4096, out_features=11008, bias=False)
          (down_proj): Linear(in_features=11008, out_features=4096, bias=False)
          (act_fn): SiLUActivation()
        )
        (input_layernorm): LlamaRMSNorm()
        (post_attention_layernorm): LlamaRMSNorm()
      )
    )
    (norm): LlamaRMSNorm()
  )
  (lm_head): Linear(in_features=4096, out_features=32000, bias=False)
)

[ydshieh]

I can reproduce (in some way): the line

> /transformers/src/transformers/utils/hub.py(418)cached_file()
-> resolved_file = hf_hub_download(

doesn't get the token when passing token to auto model's from_pretrained.

@sgugger I can take a look if you are ok with this.

[sgugger]

By all means, thanks!

[ydshieh]

@ain-soph

A fix #25083 is merged into main branch 🤗

Thank you for reporting again.