NVIDIA / NeMo-Guardrails

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
Other
4.21k stars 400 forks source link

Feat/embedding only support colang 2 #737

Closed Pouyanpi closed 2 months ago

Pouyanpi commented 2 months ago
drazvan commented 2 months ago

@Pouyanpi : I think there's a bug in both Colang 1 and 2 implementations. The threshold is meant to be for the top choice confidence. But if none is above that, and I don't have a fallback intent, then no examples are used either (because the search is performed only once with the threshold). Can you double-check? In the fallback with no intent, we should include the top 5 examples regardless of their similarity score.

Pouyanpi commented 2 months ago

In the fallback with no intent, we should include the top 5 examples regardless of their similarity score.

@drazvan, let me clarify my understanding of this:

Scenario: We are using embeddings exclusively with a search threshold and no fallback intent.