Loading ALMA-7B-R (LORA merged) through huggingface downloads Pretrained + LORA

fe1ixxu / ALMA

State-of-the-art LLM-based translation models.

MIT License

352 stars 26 forks source link

Loading ALMA-7B-R (LORA merged) through huggingface downloads Pretrained + LORA #30

Closed tranvaj closed 3 months ago

tranvaj commented 4 months ago

This started happening as of today or yesterday, before that it normally installed LORA merged and worked fine

Code im using:


from transformers import AutoModelForCausalLM
from transformers import AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("haoranxu/ALMA-7B-R", torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("haoranxu/ALMA-7B-R", padding_side='left')```

fe1ixxu commented 4 months ago

Thanks for your interest!

I guess this is because the checkpoint of the adapter is also located under ALMA-7B-R. This should not affect the performance of ALMA-R and just a way of providing the checkpoint of the adapter model.