Open xtrycatchx opened 1 month ago
Hi @xtrycatchx, I think what you are looking for concerns dialog rails. Have you followed this guide? So changing the yaml content to
yaml_content="""
instructions:
- type: general
content: |
You are an AI assistant that supports users on their inquiry about health and nutrition.
models:
- type: main
engine: openai
model: gpt-3.5-turbo-instruct
rails:
dialog:
single_call:
enabled: True
user_messages:
embeddings_only: True
"""
would resolve your issue. Try to test different configs while you are following the guide. I hope it answers your question.
Thanks @Pouyanpi .. indeed having single call mode deduce the number of calls into 1 ..
Moving forward, what if I want to use only Guardrails without the LLM, aiming for a NO-LLM call
I tried setting the yaml
to this
instructions:
- type: general
content: |
You are an AI assistant that supports users on their inquiry about health and nutrition.
rails:
input:
flows:
- politics
and the colang
to this:
define user ask politics
"What do you think about the government?"
"Which party should I vote for?"
"what are your political beliefs?"
"thoughts on the president?"
"left wing"
"right wing"
"political candidate"
"current political views"
"political figure of a country or state"
"which are companies are lobbying for liberal party?"
"are there companies lobbying for democrats?"
"what are your political views?"
"who should I vote for?"
define bot answer politics
"I'm a nutrition assistant, I don't like to talk of politics."
define flow politics
bot answer politics
stop
And at first, the output given this prompt During election, can you advice if I should choose democrats or republicans or stay neutral?
looks promising:
No LLM calls were made.
# of LLM calls: 0
{'role': 'assistant', 'content': "I'm a nutrition assistant, I don't like to talk of politics."}
BUT changing the prompt to this Is there a good vitamins or supplements you can propose for adult male aged 37?
returns the same.
No LLM calls were made.
# of LLM calls: 0
{'role': 'assistant', 'content': "I'm a nutrition assistant, I don't like to talk of politics."}
Should it be returning an empty or echoing back the user prompt?
@xtrycatchx, I think you don't need to define input rails with a flow like that, as it won't work. Removing it you will have similar output. Give it a try 👍🏻
Regarding your final suggestion, for the next release 0.10.0, it will be possible to use a fallback_intent with a defined flow. Whenever there is no specific flow to address the user's intent, it will default to the fallback_intent.
Thanks again @Pouyanpi
I just tried removing the input rails like this:
yaml_content="""
instructions:
- type: general
content: |
You are an AI assistant that supports users on their inquiry about health and nutrition.
"""
and I got an exception instead
DEBUG:asyncio:Using proactor: IocpProactor
Using proactor: IocpProactor
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Processing event: {'type': 'UtteranceUserActionFinished', 'final_transcript': 'Is there a good vitamins or supplements you can propose for adult male aged 37?'}
Processing event: {'type': 'UtteranceUserActionFinished', 'final_transcript': 'Is there a good vitamins or supplements you can propose for adult male aged 37?'}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Event :: UtteranceUserActionFinished {'final_transcript': 'Is there a good vitamins or supplements you can propose for adult male aged 37?'}
Event :: UtteranceUserActionFinished {'final_transcript': 'Is there a good vitamins or supplements you can propose for adult male aged 37?'}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Processing event: {'type': 'StartInternalSystemAction', 'uid': 'e038e9ad-c674-4146-b686-7b441103dd15', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'create_event', 'action_params': {'event': {'_type': 'UserMessage', 'text': '$user_message'}}, 'action_result_key': None, 'action_uid': 'fac837cf-4a23-4650-bda2-38c29d1c33d5', 'is_system_action': True}
Processing event: {'type': 'StartInternalSystemAction', 'uid': 'e038e9ad-c674-4146-b686-7b441103dd15', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'create_event', 'action_params': {'event': {'_type': 'UserMessage', 'text': '$user_message'}}, 'action_result_key': None, 'action_uid': 'fac837cf-4a23-4650-bda2-38c29d1c33d5', 'is_system_action': True}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Event :: StartInternalSystemAction {'uid': 'e038e9ad-c674-4146-b686-7b441103dd15', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'create_event', 'action_params': {'event': {'_type': 'UserMessage', 'text': '$user_message'}}, 'action_result_key': None, 'action_uid': 'fac837cf-4a23-4650-bda2-38c29d1c33d5', 'is_system_action': True}
Event :: StartInternalSystemAction {'uid': 'e038e9ad-c674-4146-b686-7b441103dd15', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'create_event', 'action_params': {'event': {'_type': 'UserMessage', 'text': '$user_message'}}, 'action_result_key': None, 'action_uid': 'fac837cf-4a23-4650-bda2-38c29d1c33d5', 'is_system_action': True}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Executing action :: create_event
Executing action :: create_event
INFO:nemoguardrails.actions.action_dispatcher:Executing registered action: create_event
Executing registered action: create_event
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Processing event: {'type': 'UserMessage', 'uid': '47a29088-33f3-41aa-b413-143061d37de6', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'text': 'Is there a good vitamins or supplements you can propose for adult male aged 37?'}
Processing event: {'type': 'UserMessage', 'uid': '47a29088-33f3-41aa-b413-143061d37de6', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'text': 'Is there a good vitamins or supplements you can propose for adult male aged 37?'}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Event :: UserMessage {'uid': '47a29088-33f3-41aa-b413-143061d37de6', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'text': 'Is there a good vitamins or supplements you can propose for adult male aged 37?'}
Event :: UserMessage {'uid': '47a29088-33f3-41aa-b413-143061d37de6', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'text': 'Is there a good vitamins or supplements you can propose for adult male aged 37?'}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Processing event: {'type': 'StartInternalSystemAction', 'uid': '75b0b453-0584-45fb-88d7-6a6276e2d5a0', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'generate_user_intent', 'action_params': {}, 'action_result_key': None, 'action_uid': '03e17859-5405-49d8-85c1-92d079cc8061', 'is_system_action': True}
Processing event: {'type': 'StartInternalSystemAction', 'uid': '75b0b453-0584-45fb-88d7-6a6276e2d5a0', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'generate_user_intent', 'action_params': {}, 'action_result_key': None, 'action_uid': '03e17859-5405-49d8-85c1-92d079cc8061', 'is_system_action': True}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Event :: StartInternalSystemAction {'uid': '75b0b453-0584-45fb-88d7-6a6276e2d5a0', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'generate_user_intent', 'action_params': {}, 'action_result_key': None, 'action_uid': '03e17859-5405-49d8-85c1-92d079cc8061', 'is_system_action': True}
Event :: StartInternalSystemAction {'uid': '75b0b453-0584-45fb-88d7-6a6276e2d5a0', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'generate_user_intent', 'action_params': {}, 'action_result_key': None, 'action_uid': '03e17859-5405-49d8-85c1-92d079cc8061', 'is_system_action': True}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Executing action :: generate_user_intent
Executing action :: generate_user_intent
INFO:nemoguardrails.actions.action_dispatcher:Executing registered action: generate_user_intent
Executing registered action: generate_user_intent
INFO:nemoguardrails.actions.llm.generation:Phase 1 :: Generating user intent
Phase 1 :: Generating user intent
WARNING:nemoguardrails.llm.params:Parameter temperature does not exist for NoneType
Parameter temperature does not exist for NoneType
WARNING:nemoguardrails.actions.action_dispatcher:Error while execution generate_user_intent: 'NoneType' object has no attribute 'agenerate_prompt'
Error while execution generate_user_intent: 'NoneType' object has no attribute 'agenerate_prompt'
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Processing event: {'type': 'hide_prev_turn'}
Processing event: {'type': 'hide_prev_turn'}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Event :: hide_prev_turn {}
Event :: hide_prev_turn {}
INFO:nemoguardrails.rails.llm.llmrails:--- :: Total processing took 0.01 seconds. LLM Stats: 0 total calls, 0 total time, 0 total tokens, 0 total prompt tokens, 0 total completion tokens, [] as latencies
--- :: Total processing took 0.01 seconds. LLM Stats: 0 total calls, 0 total time, 0 total tokens, 0 total prompt tokens, 0 total completion tokens, [] as latencies
No LLM calls were made.
# of LLM calls: 0
{'role': 'assistant', 'content': "I'm sorry, an internal error has occurred."}
Is my understanding correct - NeMo guardrails will take the colang configuration ( or read the .co files from path ) and encode it to a vector space and then later when a prompt is received, the prompt is then matched or calculated against the utterances and based on the flow it will behave based on how you configured it (in my case I instruct the assistant bot to answer a rejection message)?
In my case, it didn't stop. In fact it sends 3 calls to the target LLM!
Is there a way or is it possible to only use Guardrails (no LLM) using the colang flows (same for example as mine) ? I know it works for subflows, but will it work for flows as well? an Example will be appreciated.