search

ggerganov / ggml

Tensor library for machine learning
MIT License
11.26k stars 1.05k forks source link

[Feature Request] Support instructor-xl (for text embeddings) #109

Open sroussey opened 1 year ago

sroussey commented 1 year ago

It ranks at the top in the hugging-face leaderboard:

https://huggingface.co/spaces/mteb/leaderboard

Here is more info:

https://huggingface.co/hkunlp/instructor-xl

It is a little different... you add an instruction context separately from the thing you want to create embeddings. Quite versatile; obviates the need for many specialized models.

skeskinen commented 1 year ago

It's not instructor-xl, but I've done ggml implementation of BERT for sentence embeddings here: https://github.com/skeskinen/bert.cpp

ggerganov commented 1 year ago

@skeskinen

Really well done! Feel free to add a link to your project in ggml’s README Adding batching support to ggml is getting higher on the TODO list. Now with bert.cpp we’ll have something to easily test with!

skeskinen commented 1 year ago

Thanks! Adding support for batching sounds exiting :+1: It'll be nice to see what kind of speedup it will give and if bert.cpp will be faster than pytorch(cpu) in batched head to head.