search

deadbits / vigil-llm

⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs
https://vigil.deadbits.ai/
Apache License 2.0
277 stars 33 forks source link

Change transformer model to deepset #22

Closed deadbits closed 10 months ago

deadbits commented 10 months ago

deepset/deberta-v3-base-injection

https://github.com/langchain-ai/langchain/blob/master/libs/experimental/langchain_experimental/prompt_injection_identifier/hugging_face_identifier.py

deadbits commented 10 months ago

This seems to perform similarly (poorly) to the last model I tried.. but at this point I've already made the changes to use the deepset model. 6 in one hand, half dozen in the other situation.

I'll keep the deepset model for now as a back-up check, but it is so prone to false positives I'm considering removing it entirely.

https://github.com/deadbits/vigil-llm/commit/5489c55b483c56c26faa19d741577aa5071c6386