Cannot run eval with local model directory

[luquitared]

Hi. Thank you for your hard work!

I am trying to run bigcode/starcoder model on a server. I downloaded the huggingface repo with git and transferred the folder to the server.

To be extra sure I wasn't passing the path wrong, I modified main.py:

    parser.add_argument(
        "--model",
        default="/home/ubuntu/.cache/huggingface/hub/models--bigcode--starcoder/snapshots/8a57e3930912e5d22ddc4d5f46b4b99f169afbe9",
        help="Model to evaluate, provide a repo name in Hugging Face hub or a local path",
    )

Once I run python main.py I get:

─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/ubuntu/star-coder/bigcode-evaluation-harness/main.py:230 in <module>                       │
│                                                                                                  │
│   227                                                                                            │
│   228                                                                                            │
│   229 if __name__ == "__main__":                                                                 │
│ ❱ 230 │   main()                                                                                 │
│   231                                                                                            │
│                                                                                                  │
│ /home/ubuntu/star-coder/bigcode-evaluation-harness/main.py:176 in main                           │
│                                                                                                  │
│   173 │   │   │   │   f"Non valid precision {args.precision}, choose from: fp16, fp32, bf16"     │
│   174 │   │   │   )                                                                              │
│   175 │   │   print(f"Loading tokenizer and model (in {args.precision})")                        │
│ ❱ 176 │   │   model = AutoModelForCausalLM.from_pretrained(                                      │
│   177 │   │   │   args.model,                                                                    │
│   178 │   │   │   revision=args.revision,                                                        │
│   179 │   │   │   torch_dtype=dict_precisions[args.precision],                                   │
│                                                                                                  │
│ /home/ubuntu/.local/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py:434 in  │
│ from_pretrained                                                                                  │
│                                                                                                  │
│   431 │   │   ]                                                                                  │
│   432 │   │   hub_kwargs = {name: kwargs.pop(name) for name in hub_kwargs_names if name in kwa   │
│   433 │   │   if not isinstance(config, PretrainedConfig):                                       │
│ ❱ 434 │   │   │   config, kwargs = AutoConfig.from_pretrained(                                   │
│   435 │   │   │   │   pretrained_model_name_or_path,                                             │
│   436 │   │   │   │   return_unused_kwargs=True,                                                 │
│   437 │   │   │   │   trust_remote_code=trust_remote_code,                                       │
│                                                                                                  │
│ /home/ubuntu/.local/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py:8 │
│ 29 in from_pretrained                                                                            │
│                                                                                                  │
│   826 │   │   │   )                                                                              │
│   827 │   │   │   return config_class.from_pretrained(pretrained_model_name_or_path, **kwargs)   │
│   828 │   │   elif "model_type" in config_dict:                                                  │
│ ❱ 829 │   │   │   config_class = CONFIG_MAPPING[config_dict["model_type"]]                       │
│   830 │   │   │   return config_class.from_dict(config_dict, **unused_kwargs)                    │
│   831 │   │   else:                                                                              │
│   832 │   │   │   # Fallback: use pattern matching on the string.                                │
│                                                                                                  │
│ /home/ubuntu/.local/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py:5 │
│ 36 in __getitem__                                                                                │
│                                                                                                  │
│   533 │   │   if key in self._extra_content:                                                     │
│   534 │   │   │   return self._extra_content[key]                                                │
│   535 │   │   if key not in self._mapping:                                                       │
│ ❱ 536 │   │   │   raise KeyError(key)                                                            │
│   537 │   │   value = self._mapping[key]                                                         │
│   538 │   │   module_name = model_type_to_module_name(key)                                       │
│   539 │   │   if module_name not in self._modules:                                               │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
KeyError: 'gpt_bigcode'

Any idea on how to fix this issue? Had trouble authenticating my huggingface that is why I am trying to load a local model.

[luquitared]

Can confirm that I am using transformers==4.28.1

[loubnabnl]

Can you try uninstalling and re-installing transformers again the error indicates not finding the GPTBigCode architecture.

Since it's an issue with loading the model, to debug you can try running this and make sure the model loads

from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(YOUR_PATH)

[loubnabnl]

Did you fix the issue?

[loubnabnl]

closing as issue is inactive