search

SeanLee97 / AnglE

Train and Infer Powerful Sentence Embeddings with AnglE | 🔥 SOTA on STS and MTEB Leaderboard
https://arxiv.org/abs/2309.12871
MIT License
493 stars 33 forks source link

Error extracting angle-llama vector #57

Closed Davido111200 closed 8 months ago

Davido111200 commented 8 months ago

Hi Sean,

Thanks for the amazing work! I notice that there might be a small bug in newer versions of the code resulting in a device error when using angle-llama to get embeddings. I downgrade the version to 0.3.0 and the problem disappears.

To reproduce the error, simply execute the code given in Angle-llama instructions

Could you take a quick look at the problem? Thanks !

SeanLee97 commented 8 months ago

hi @Davido111200 , I checked the latest version (v0.3.8) and found it works. Could you send me your inference code?

from angle_emb import AnglE

model = AnglE.from_pretrained('NousResearch/Llama-2-7b-hf', pretrained_lora_path'SeanLee97/angle-llama-7b-nli-v2').cuda()

model.encode({'text': 'hello world'})
Davido111200 commented 8 months ago

Hi @SeanLee97 . Thanks for your reply. My inference code looks almost exactly like yours, but I did not transfer the model to CUDA. It seems to me that is the only problem.