We need a standardized, thought-out flow for when the agent receives insufficient information to generate a proper response. Additionally, the agents need to be super sensitive to insufficient info, as hallucinations with insufficient info could be very bad. I have a temporary solution for this in EventEditor, instructing the model to return an error json if no given event clearly matches the prompt:
"IMPORTANT: IF THE USER INPUT IS AT ALL UNCLEAR, OR DOES NOT CLEARLY MATCH UP TO AN EVENT FROM THE LIST, RETURN A JSON BRIEFLY BUT AMICABLY DETAILING THE ERROR. This should be the default behavior, i.e. most instructions should not correspond to any event given."
The process_response method then handles this case.
But this doesn't manifest in clear follow up question (which should be handled by the central agent).
We need a standardized, thought-out flow for when the agent receives insufficient information to generate a proper response. Additionally, the agents need to be super sensitive to insufficient info, as hallucinations with insufficient info could be very bad. I have a temporary solution for this in
EventEditor, instructing the model to return an error json if no given event clearly matches the prompt: "IMPORTANT: IF THE USER INPUT IS AT ALL UNCLEAR, OR DOES NOT CLEARLY MATCH UP TO AN EVENT FROM THE LIST, RETURN A JSON BRIEFLY BUT AMICABLY DETAILING THE ERROR. This should be the default behavior, i.e. most instructions should not correspond to any event given." Theprocess_responsemethod then handles this case. But this doesn't manifest in clear follow up question (which should be handled by the central agent).