OSError when loading the model

JavierCastellD commented 1 month ago

Hello,

I have followed the steps to install LLM2Vec from the GitHub repository and I'm trying to run the example code found on HuggingFace: https://huggingface.co/McGill-NLP/LLM2Vec-Mistral-7B-Instruct-v2-mntp.

However, when I execute it, I get the following error when loading the AutoModel.from_pretrained(...):

Traceback (most recent call last):
  File "/home/jcastell/snomedEmbeddings/create_dict_llm2vec.py", line 27, in <module>
    model = AutoModel.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jcastell/miniforge3/envs/llm2vec_env/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 558, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jcastell/miniforge3/envs/llm2vec_env/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3451, in from_pretrained
    raise EnvironmentError(
OSError: mistralai/Mistral-7B-Instruct-v0.2 does not appear to have a file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt or flax_model.msgpack.

The supposedly missing files are in https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/tree/main so I am not entirely sure why I am getting that error or how could I solve it.

vaibhavad commented 4 weeks ago

Hi @JavierCastellD,

Is the peft library also installed in your environment? The loading only works if both peft and transformers library is installed.

Alternatively, you can also load using llm2vec library, as documented here.

Let me know if you have any further questions.

JavierCastellD commented 2 weeks ago

Hi.

Both libraries were installed. Apparently the problem was that I did not properly set the huggingface access token permissions.

So now it is correctly working.

Thanks!

McGill-NLP / llm2vec

OSError when loading the model #90