search

huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
135.38k stars 27.09k forks source link

Translation model M2M100 uses 2 models in cache (from version 4.46.0) #34731

Open harelfar2 opened 1 week ago

harelfar2 commented 1 week ago

System Info

I'm using facebook/m2m100_418M translation model. From version 4.46.0 it downloads another model which wieghts ~2 GB. I'm using python 3.11, in ubuntu

Who can help?

@ArthurZucker

Information

Tasks

Reproduction

import torch
from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer

model = M2M100ForConditionalGeneration.from_pretrained(
    "facebook/m2m100_418M",
    torch_dtype=torch.float16,
).to("cpu").eval()

token = M2M100Tokenizer.from_pretrained("facebook/m2m100_418M")

encoded_text = token("my name is earl", return_tensors="pt")
encoded_text = encoded_text.to("cpu")

target_lang_id = token.get_lang_id("he")
generated_tokens = model.generate(**encoded_text, forced_bos_token_id=target_lang_id)
print(generated_tokens)

Expected behavior

The models are being put in /home/ubuntu/.cache/huggingface/hub/models--facebook--m2m100_418M/ until version 4.46.0 there was this hierarchy: snapshots/55c2e61bbf05dfb8d7abccdc3fae6fc8512fd636 which contained 7 files (one of them is the model itself pytorch_model.bin - ~2 GB). From version 4.46.0, there is a new dir: snapshots/791dc1c6d300846c9a747d4bd11fcc7f369b750e, there is one file in there: model.safetensors, which is a soft link to another heavy ~2GB file in blobs dir.

Can you please resolve it and make it download and use only one model file? this usage is very wasteful.

Thanks!

LysandreJik commented 1 week ago

Hey @harelfar2, we're slowly moving towards deprecating support of pytorch_model.bin files as several security issues have been found with these files.

As such, when there is a model.safetensors file available as an equivalent file, we now default to downloading and using this file. As a result, you should be able to safely remove the pytorch_model.bin file in your cache and the model should still work as expected.

Please let me know if you run into any issues