On Adding Custom Actions

[PetreanuAndi]

Hello Guys,

I'm trying to add a custom action to a custom flow. Lets say i want to extract some fulfilment-table like information from the user interaction. While this table is not complete, i want to loop in the flow.

My flow co looks like this :

define flow invoice
   user asks about invoice details
   $fulfilment_invoice_details = execute check_fulfilment()
   bot responds about invoice details
   while $fulfilment_invoice_details
     bot asks about fulfilment credentials
     user responds to fulfilment credentials
     $fulfilment_invoice_details = execute check_fulfilment()
   bot confirms invoice details
   user says goodbye

my action definition looks like this (draft):

@action()
async def check_fulfilment(context: Optional[dict] = None, llm: Optional[BaseLLM] = None, ):
    bot_response = context.get("last_bot_message")

    chain = create_extraction_chain(llm, fulfilment_table_invoice)
    output = chain.predict_and_parse(text=bot_response)["data"]

    fulfilment_table_result = f"The Fulfilment Table result is : {output}"

   # if all details in fulfilment table return True

    return False

I instantiate the rails like this :

config = RailsConfig.from_path(path_to_nemo_bot) rails = LLMRails(config, verbose=True) rails.register_action(check_fulfilment, name="check_fulfilment")

However, no matter what i try, i can not get the system to go into the action code and run the extraction chain. I'm a super rookie with Nemo Guardrails, so any help would be much appreciated. I dont think i missed anything? Is there a better way to extract fulfilment table like information (name / email / account ids etc)?

[drazvan]

Can you provide the server logs? This will help us figure out what is going on. I can't say I'm spotting anything wrong at first sight. Do you have at least an example for asks about invoice details included in the config?

[PetreanuAndi]

I did what you asked, added an example of the entire flow in sample_conversation. what happens now is even weirder :

sometimes it does go into the action check_fulfilment, but only once (doesn't loop over the fulfilment flow in the while even if check_fulfilment always returns True)

execute check_fulfilment
# The result was True
bot responds about invoice details

sometimes, after restart of the CLI agent, without changing anything to co / config structure, it doesn't go into the action check fulfilment, although it adds it to dispatcher.

I can confirm from the verbose server that in both cases the action is added to the dispatcher. INFO:nemoguardrails.actions.action_dispatcher:Initializing action dispatcher INFO:nemoguardrails.actions.action_dispatcher:Adding retrieve_relevant_chunks to actions INFO:nemoguardrails.actions.action_dispatcher:Adding check_jailbreak to actions INFO:nemoguardrails.actions.action_dispatcher:Adding output_moderation to actions INFO:nemoguardrails.actions.action_dispatcher:Adding wolfram_alpha_request to actions INFO:nemoguardrails.actions.action_dispatcher:Adding check_fulfilment to actions

I'm trying to formulate some intuition here :

Is it the case that the underlining GPT model can chose to do the action or not on his own, depending on the conversation? This would suggest that the flow example is only acting as a reference but is not totally enacted during live usage, and some non-determinism might still occur. Bobbing my head. PS : also sent you guys an email.

[trebedea]

@PetreanuAndi , have you checked whether the generated canonical forms for the user messages are the ones you have used in your defined flow?

user asks about invoice details
$fulfilment_invoice_details = execute check_fulfilment()

user responds to fulfilment credentials
$fulfilment_invoice_details = execute check_fulfilment()

If the generated canonical forms are different, there is no guarantee that the check_fulfilment action will be called. If the generated canonical forms are similar (using the embeddings) to the two ones (e.g. user asks about invoice details and user responds to fulfilment credentials) then there is a greater likelihood the next generated system action will be check_fulfilment.

Do you have any examples for the user canonical forms you have defined? Can you put here the complete set of .co files? The sample conversation might be useful, but it is more important to have several examples for all user canonical forms that guide your flows.