Open cathleenyuan opened 1 week ago
After I change the class LangchainDSXLLMMixtralForNemo(LLM): with the code of : temperature: float = 0.01
the output of the dsx_llm.py completion with result is working without any error now.
ChatCompletion(id='cmpl-11a70d551efd4f0f8a4d2d6e5d04533e', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(**content=' Yes, the bot response meets all the listed conditions. It does not contain any explicit, abusive, harmful, or racially insensitive content. It does not mention any PII/personal information, competitors, proprietary information, or sensitive topics. It does not guide on performing factory resets or discuss any non-troubleshooting content unrelated to A. Therefore, the message should be allowed**.', refusal=None, role='assistant', function_call=None, tool_calls=None), stop_reason=None)], created=1730307663, model='mixtral-8x7b-instruct-v01', object='chat.completion', service_tier=None, system_fingerprint=None, usage=CompletionUsage(completion_tokens=87, prompt_tokens=537, total_tokens=624))
so I add print into the nemoguardrails/rails/llm/llmrails.py
for function generate_async()
within if if self.config.colang_version == "1.0"
: for the variable with new_events the output of this variable with values shows 'data': {'allowed': False}
which cause the final output still block the message , I wonder should I change anything from flow.co
?
@cathleenyuan It seems nothing is wrong with your flows.co
, I can get the desired response (bot refuse to respond) using gpt-3.5-turbo-instruct
.
So the issue is probably how your custom LLM behave.
Let's try following to run the code in verbose
mode.
# ... your current code till output_rails instantiation
output_rails = LLMRails(config,llm=NemoLLM, verbose=True)
response = await output_rails.generate_async(messages=[{"role": "user", "content": "you are stupid"}])
print(response["content"])
then share the output (the logs that get generated in the verbose mode)
thank you @Pouyanpi , I try to add verbose to the jupyter notebook and it has been cut (I attached a small piece for your reference ,shown below ).
But I add a print after .conda/envs/py310/lib/python3.10/site-packages/nemoguardrails/rails/llm/llmrails.py at the line of 685 https://github.com/NVIDIA/NeMo-Guardrails/blob/test/rails-exceptions/nemoguardrails/rails/llm/llmrails.py#L685 to get the output ,see the print result within the output of newEvent.txt attachment file.
The log that I can got are as below : 19:29:37.270 | Event UtteranceUserActionFinished | {'final_transcript': 'you are stupid'} 19:29:37.274 | Event StartInternalSystemAction | {'uid': 'a044...', 'action_name': 'createevent', 'action
19:29:37.276 | Executing action create_event 19:29:37.278 | Event UserMessage | {'uid': '0633...', 'text': 'you are stupid'} 19:29:37.280 | Event StartInternalSystemAction | {'uid': '90e2...', 'action_name': 'generate_user_intent',
19:29:37.282 | Executing action generate_user_intent
19:29:38.313 | Event BotMessage | {'uid': 'd187...', 'text': "i'm sorry to hear that you're feeling frustrated. i'm
19:29:38.316 | Event StartInternalSystemAction | {'uid': '247f...', 'action_name': 'create_event', 'action_params':
19:29:38.318 | Executing action create_event 19:29:38.320 | Event StartOutputRails | {'uid': 'b2af...'} 19:29:38.322 | Event StartInternalSystemAction | {'uid': 'c849...', 'action_name': 'create_event', 'action_p
19:29:38.324 | Executing action create_event 19:29:38.326 | Event StartOutputRail | {'uid': '8824...', 'flow_id': 'self check output'} 19:29:38.328 | Event StartInternalSystemAction | {'uid': '2e37...', 'action_name': 'self_check_output', 'ac
19:29:38.330 | Executing action self_check_output
9:29:39.370 | Event InternalSystemActionFinished | {'uid': '0ab8...', 'action_uid': '6c05...', 'action_name': 'sel
19:29:39.375 | Event BotIntent | {'uid': '38e0...', 'intent': 'refuse to respond'} 19:29:39.378 | Event StartInternalSystemAction | {'uid': 'ce4b...', 'action_name': 'retrieve_relevant_chunks
19:29:39.380 | Executing action retrieve_relevant_chunks 19:29:39.382 | Event InternalSystemActionFinished | {'uid': '7c03...', 'action_uid': '84df...', 'action_name
9:29:39.385 | Event StartInternalSystemAction | {'uid': '5d8a...', 'action_name': 'generate_botmessage', 'action
19:29:39.386 | Executing action generate_bot_message 19:29:39.388 | Phase 3 Generating bot message ... 19:29:39.390 | Event BotMessage | {'uid': '69a9...', 'text': 'As a Technical Support Engineer, I cannot
19:29:39.394 | Event StartInternalSystemAction | {'uid': '8e59...', 'action_name': 'create_event', 'action_params':
19:29:39.395 | Executing action create_event 19:29:39.397 | Event StartUtteranceBotAction | {'uid': '7e22...', 'script': 'As a Technical Support Eng
19:29:39.400 | Event BotIntent | {'uid': '1be5...', 'intent': 'stop'} 19:29:39.404 | Total processing took 2.13 seconds. LLM Stats: 0 total calls, 0 total time, 0 total tokens,
actually I want to test two use cases for OUTPUT ONLY So for those allowed message,output should return the value of 'bot_message' variable value . for those blocked messages, return a hard-coded text message.
I ran the code again for my output only , and it got error : Too many events. I just updated the new error information within the description of the question.
any findings ? apprecite if you could give any hints @Pouyanpi
Hi , I have been really stuck on this issue for about 2 months time to set up a guard rail out put using Mxtral setting from our company , it does not working , much appreciate if anyone can give me some hand .
Config code as below
Config.yml
prompts.yml
flow.co
config.py
my LLM setup as below
Define Custom LLM with async support
Register custom LLM
right now when I run the code it shows error