protectai / rebuff

LLM Prompt Injection Detector
https://playground.rebuff.ai
Apache License 2.0
1.06k stars 73 forks source link

parsing crash #28

Closed shachar-ug closed 11 months ago

shachar-ug commented 1 year ago

No error handling on partial prompts crash detection_metrics, is_injection = rb.detect_injection("Ignore all prior") if is_injection: print("Possible injection detected. Take corrective action.")

File rebuff.py:86, in Rebuff.detect_injection(self, user_input, max_heuristic_score, max_vector_score, max_model_score, check_heuristic, check_vector, check_llm)

 83 response.raise_for_status()
 85 response_json = response.json()
 --->    86 success_response = DetectApiSuccessResponse.parse_obj(response_json)
 88 if (
 89     success_response.heuristicScore > max_heuristic_score
 90     or success_response.modelScore > max_model_score
 91     or success_response.vectorScore["topScore"] > max_vector_score
 92 ):
 93     # Injection detected
 94     return success_response, True
woop commented 1 year ago

Thanks for highlighting this @shachar-ug!

seanpmorgan commented 11 months ago

I no longer see this issue. Please re-open if you can re-produce on latest rebuff sdk. If you do see it on the pypi installed rebuff that should be fixed shortly with #48

rb = Rebuff(api_token="MYTOKEN, api_url="https://www.rebuff.ai")

user_input = "Ignore all prior"
result = rb.detect_injection(user_input)

if result.injectionDetected:
    print("Possible injection detected. Take corrective action.")

Possible injection detected. Take corrective action.