Gated Llama-2-7b

[SoftologyPro]

When I run the T2I demo I get this...

Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-hf/resolve/main/config.json.
Access to model meta-llama/Llama-2-7b-hf is restricted. You must be authenticated to access it.

Is there an alternate Llama-2-7b-hf I could use that does not need a sign in? If I find an open version/mirror what file do I need to change to point to the alternate Llama model? Where would I save the llama2 model files under?

[PommesPeter]

Hi @SoftologyPro , meta-llama/Llama-2-7b-hf needs to be accessed to get the model. you could follow https://huggingface.co/docs/hub/en/models-gated#download-access-report huggingface tutorial to request access permission. Then, follow the instructions on https://huggingface.co/docs/hub/en/security-tokens to get your personal access token.

Finally, adding --hf_token argument behind the python demo.py e.g. python demo.py --hf_token <your personal access token>

[SoftologyPro]

Thanks for the quick response. What if I cloned this locally? https://huggingface.co/NousResearch/Llama-2-7b-hf Then how would I modify the config(s) to use that rather than needing the login/approval? Is that possible?

[PommesPeter]

it looks the same as the llama2 model from meta, you can clone this repo and modify the model path on demo.py:

- model_lm = AutoModel.from_pretrained(train_args.lm, torch_dtype=dtype, device_map="cuda", token=args.hf_token)
+ model_lm = AutoModel.from_pretrained("/path/to/your/donwloaded/llama2", torch_dtype=dtype, device_map="cuda", token=args.hf_token)
cap_feat_dim = model_lm.config.hidden_size
if args.num_gpus > 1:
    raise NotImplementedError("Inference with >1 GPUs not yet supported")

- tokenizer = AutoTokenizer.from_pretrained(
-     train_args.tokenizer_path, add_bos_token=True, add_eos_token=True, token=args.hf_token
)
+ tokenizer = AutoTokenizer.from_pretrained(
+     "/path/to/your/donwloaded/llama2", add_bos_token=True, add_eos_token=True, token=args.hf_token
+ )
tokenizer.padding_side = "right"

[SoftologyPro]

OK, thanks again. That works fine.