search

fe1ixxu / ALMA

State-of-the-art LLM-based translation models.
MIT License
395 stars 29 forks source link

Fail to load ALMA-13B #50

Closed wygao8 closed 1 month ago

wygao8 commented 3 months ago

Hi I used huggingface-cli to download both haoranxu/ALMA-13B-R and ALMA-13B in the same cache directory.

I can load ALMA-13B-R successfully with the following command:

model = AutoModelForCausalLM.from_pretrained(
    "haoranxu/ALMA-13B-R", 
    cache_dir=my_cache_dir, 
    torch_dtype=torch.float16, 
    device_map="auto")

but failed to load ALMA-13B

model = AutoModelForCausalLM.from_pretrained(
    "haoranxu/ALMA-13B", 
    cache_dir=my_cache_dir, 
    torch_dtype=torch.float16, 
    device_map="auto")

The error log show as follows:

urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with 
url: /haoranxu/ALMA-13B/resolve/main/model.safetensors.index.json (Caused by ConnectTimeoutError(<urllib3.connecti
on.HTTPSConnection object at 0x7f4ef4913290>, 'Connection to huggingface.co timed out. (connect timeout=10)'))    

During handling of the above exception, another exception occurred:     

requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded wit
h url: /haoranxu/ALMA-13B/resolve/main/model.safetensors.index.json (Caused by ConnectTimeoutError(<urllib3.connec
tion.HTTPSConnection object at 0x7f4ef4913290>, 'Connection to huggingface.co timed out. (connect timeout=10)'))
wygao8 commented 3 months ago

It seems to be a bug for transformers==4.41.2, because I can load ALMA-13B with transformers==4.39.3.

I have submitted an issue to the official transformers repo (https://github.com/huggingface/transformers/issues/31552).