search

deadbits / vigil-llm

⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs
https://vigil.deadbits.ai/
Apache License 2.0
277 stars 33 forks source link

Look into using fastembed #19

Closed deadbits closed 10 months ago

deadbits commented 10 months ago

Fastembed lets you run quantized models for embeddings. With this we could avoid the pytorch requirement for sentence transformers, and just use fastembed for all the same models.

https://github.com/qdrant/fastembed

deadbits commented 10 months ago

Limited on which models can be used; skipping for now