Closed djokester closed 3 weeks ago
Hey @djokester , thanks for the suggestion. I totally agree that this scanner will throw false-positives if not given more context about the prompt. Therefore, it's definitely a good idea to add a prompt together with the output there.
Hey @djokester , thank you for the idea. We added this feature.
The scan function within the bias.py file does not utilize the provided prompt, which creates an expectation that the LLM would need to respond by quoting the context of the prompt for this scenario to work effectively.
https://github.com/protectai/llm-guard/blob/5cd2c5d8b3928296d85755ba065ac16cd6c2b361/llm_guard/output_scanners/bias.py#L85
Bias detection is highly context-dependent, and without the context of the prompt, it could misidentify or overlook bias, as it might pertain to a host of other factors.
Unlike other metrics such as toxicity or sentiment, bias cannot be solely detected from the language of the output. Context is crucial for identifying bias related to race, gender, sexual orientation, etc., as these elements might appear as mere identifying points without the context of the prompt.
Outputs without Context
Outputs with Context