search

NVIDIA / NeMo-Guardrails

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
Other
4.02k stars 367 forks source link

Error evaluating '$generation_options.rails.input': Attribute 'rails' does not exist in expression 'var_generation_options.rails.input' #574

Closed johnyoonh closed 2 months ago

johnyoonh commented 3 months ago

The define flow process user input has if $generation_options is None or $generation_options.rails.input: which should prevent input rails from running if I have omitted the rail in the options.rails:

In the JSON payload I have:

"options": {
        "rails": [
            "output"
        ]
    },

Putting a breakpoint on generate_async function in llmrails.py:544, I see that rails field has been populated properly: {'llm_output': False, 'llm_params': None, 'log': {'activated_rails': False, 'colang_history': False, 'internal_events': False, 'llm_calls': False}, 'output_vars': None, 'rails': {'dialog': False, 'input': False, 'output': True, 'retrieval': False}}

But this condition seems to get ignored. So when I have chaged it to: ir $generation_options.rails.input:, I am getting the follwing error:

lib/python3.12/site-packages/nemoguardrails/colang/v1_0/runtime/eval.py", line 71

 in eval_expression
    raise Exception(f"Error evaluating '{expr}': {str(ex)}")
Exception: Error evaluating '$generation_options.rails.input': Attribute 'rails' does not exist in expression 'var_generation_options.rails.input'

The interpreter for colang doesn't seem to be working properly either in v0.8.1 or v0.9.0 . Is there a way to making sure that var_genrations_options.rails is set properly?

johnyoonh commented 3 months ago

this was a due to a bug I introduced myself by changing the context update flow, will create a separate issue explaining the necessity of passing relevant_chunks as an input