guardrails-ai / guardrails

Adding guardrails to large language models.
https://www.guardrailsai.com/docs
Apache License 2.0
4.16k stars 317 forks source link

[bug] Validating bias check to "The movie was great!" is classified as biased #1169

Open OrenRachmil opened 1 week ago

OrenRachmil commented 1 week ago

Describe the bug I followed the usage example for applying the bias_check validator, but I encountered an unexpected result. The validation failed with the following error message:

Validation failed for field with errors: The original response contains potentially biased messages:
The movie was great!
 (Message scores: 0.9892271757125854)

To Reproduce Run the example code from the documentation

Expected behavior A clear and concise description of what you expected to happen.

Library version: Version 0.60

Additional context Python Version: 3.12 OS: Windows 11

AlejandroEsquivel commented 3 days ago

Seems to be an issue with the underlying model due to a recent update

Tracking with the issue: https://github.com/guardrails-ai/bias_check/issues/3