search

protectai / rebuff

LLM Prompt Injection Detector
https://playground.rebuff.ai
Apache License 2.0
1.09k stars 78 forks source link

ChromaDB Support for Python SDK #110

Open mehrinkiani opened 8 months ago

mehrinkiani commented 8 months ago

This PR adds Chroma DB support for Python SDK.

ristomcgehee commented 7 months ago

Just wanted to let you know that I've been busy lately with my day job and probably won't be able to get to this for at least a week.

mehrinkiani commented 7 months ago

Sounds good, and thank you for letting me know @ristomcgehee!

mehrinkiani commented 7 months ago

Thank you @ristomcgehee for the review, and suggestions! I have tried to incorporate most of them. Also using chromadb.HttpClient makes sense to me, I am going to work on setting a remote server. Thought would share the update on the PR for now

mehrinkiani commented 7 months ago

Thank you @ristomcgehee, I have now added Docker files for Chroma server. Though not sure why the JS and Python tests (integration tests) are failing. They are detecting prompt injection when there is none

mehrinkiani commented 6 months ago

A couple ways we could address that:

  • Set the temperature to 0 when calling OpenAI
  • Retry the tests multiple times when they fail

Thank you for the suggestions. I have tried rerunning the tests multiple times, and have also set temperature to 0 when calling OpenAI, thought don't think it is helping much.

Python SDK tests are also failing because of connection error with chroma server when they do pass locally. I will continue to debug this, but if you have any suggestion please do share.