Neural-Dragon-AI / Cynde

A Framework For Intelligence Farming
Apache License 2.0
12 stars 0 forks source link

add support for local embeddings with hf transformers #3

Open furlat opened 4 months ago

furlat commented 4 months ago

https://huggingface.co/jinaai/jina-embeddings-v2-base-en https://huggingface.co/nomic-ai/nomic-embed-text-v1

pszemraj commented 4 months ago

here's a gist I made covering usage of the nomic-text-v1 model with ONNX - might help later.

there are likely further optimizations within onnx (or even openvino) to be made but I can get 2ish docs/sec at 8192 ctx on CPU, could be worse