search

ggerganov / llama.cpp

LLM inference in C/C++
MIT License
67.29k stars 9.67k forks source link

Support for torch's Longstorage for Sentance Similiary tasks #5881

Closed gevzak closed 5 months ago

gevzak commented 8 months ago

Now since BERT/SBERT models are supported by LLamaCPP, it would be great to support models such as "all-mpnet-base-v2". Currently, when attempting to convert the modelm cover.py throws an error "KeyError: ('torch', 'LongStorage')." in e "convert.py", line 832, in find_class.

If the model had to be supported with the latest changes, I would switch the ticket type to Bug, but if not, the motivation behind this is the model is pretty popular and giving great results in Sentence Similarity tasks. If anyone has an idea what steps would be required to support this model and the effort required, would be happy to hear that and eventually contribute.

hiepxanh commented 8 months ago

I have the same issue, that will be the nice support to unlock embedding feature

ub3rb3457 commented 7 months ago

bump

jaasperhelin commented 7 months ago

Would be great if the all-mpnet-base-v2 model would be enabled. It works pretty well for German language and a combination of German & English text.

ghost commented 6 months ago

Try to use convert-hf-to-gguf.py instead of convert.py. It worked with gte-base-zh.

github-actions[bot] commented 5 months ago

This issue was closed because it has been inactive for 14 days since being marked as stale.