code-kern-ai / refinery

The data scientist's open-source choice to scale, assess and maintain natural language data. Treat training data like a software artifact.
https://www.kern.ai
Apache License 2.0
1.39k stars 66 forks source link

Implement support for Qdrant's fastembed #274

Open FelixKirschKern opened 10 months ago

FelixKirschKern commented 10 months ago

Is your feature request related to a problem? Please describe. Qdrant launched a new embedder that enables fast and lightweight embedding creation. The embedder performs well on CPU and the embeddings achieve good results in benchmarks.

Here is the article describing the new model.

Describe the solution you'd like First, we need to test the embedder ourselves to determine if it is a good extension to our current supported embedders. If so, we would implement support for the model in refinery.