Closed pbernabeup closed 1 month ago
Have you solved it ?
I have managed to find a workaround for this issue.
If your local path coincides with the name of the official model, the HookedTransformer.from_pretrained() detects it correctly and loads it. So in this case I solved it using the following code:
hook_trf = HookedTransformer.from_pretrained(
model_name="roneneldan/TinyStories-1Layer-21M",
local_files_only=True,
)
Where "roneneldan/TinyStories-1Layer-21M" is both the path of the model in my system and in HuggingFace.
Additionally, if you intend to use a fine-tuned version of a supported model, you can rename the local folder as the model's official name and it will load correctly. For example, I downloaded Aloe, a medical fine-tune of Llama3, and changed the folder names from "HPAI-BSC/Llama3-Aloe-8B-Alpha" to "meta-llama/Meta-Llama-3-8B" and loaded it using this code:
hook_trf = HookedTransformer.from_pretrained(
model_name="meta-llama/Meta-Llama-3-8B",
local_files_only=True,
)
This can also be done with git clone <finetune_hf_repo_name> <supported_hf_repo_name>
.
When dealing with TransformerLens, it's best to keep the model names the official names used on hugging face. What I gather from your code is that you have a local repo where you reorganize your models? If I am wrong on that let me know. If that is the case, a little info... TransformerLens takes in what you pass through as the model name to load in a bunch of different things needed to run a model within TransformerLens. Most of the loading of the actual model is offloaded to transformers. Before we get to that point, TransformerLens will try to guess what the official name of the model is. It does that by checking a map of aliases to see if a specific alias has been configured, which means that if you have a custom path that differs from the official name, you are greatly lowering the possibility of it figuring out what the actual model is, since someone would have had to previously configure the model name as you have it to point to the official model name. For all of these reasons, it is almost always best to load the model from the official name, and to keep things as is. If you want to add an alias, I can show you how to do that, but I am going to close this unless you want to do that.
Question
I am attempting to run a model in an offline environment using the following code:
If I set model_name to the path of the local clone of the repo, it attempts to convert it to a valid official name and throws this error:
However, if I set model_name to the name of the HF repo the model tries to access HuggingFace and throws the following error:
How can I fix this?