search

NVIDIA / NeMo-Guardrails

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
Other
4.21k stars 400 forks source link

AlignScore factcheck.co is not working when accuracy is lower than threshold #736

Open Ryan-ZL-Lin opened 2 months ago

Ryan-ZL-Lin commented 2 months ago

How to reproduce the issue?

  1. setup AlignScore server and use large model

  2. Reference Configuration: https://github.com/NVIDIA/NeMo-Guardrails/blob/develop/examples/configs/rag/fact_checking config: Image

    rails: factcheck.co and general.co

Who could help? @drazvan

Description

Hi I tried out the whole setup, and it seemed work well initially. I got an successful testing result by making bot to use "inform answer unknown" when the accuracy from AlignScore was lower than 0.4.

However, when I tried to adjust factcheck.co to have different bot response, it's not working as expected. For example, I changed the threshold to make the Bot to use "inform answer potentially inaccurate" as follows, but I still got "inform answer unknown" even the score is above 0.4.

Does one who could provide some tips to better address this issue?

Image

Image

here is what I found in colag_history

Image