xrx state machine draft

[mprast]

XRX State Machine

The purpose of this PR is to add a backing state machine to the xrx reasoning agent. A sample state machine has been fully integrated into shopify-app. With this change the agent (mostly) appears to be able to:

• Understand the state it's in and infer its current objective
• Understand what states it can transition into
• Transition to the next state when appropriate
• Guide the user back to the current objective if they try to go out of bounds
• Switch flows if the user indicates they want to do something different

Testing

No special setup is needed for the state machine; just pull the branches down for xrx-sample-apps and xrx-core and play around with shopify-app as usual. The agent will log what state it's in and will use a 'transition-state' node to transition when appropriate.

A flow is a graph of steps. Each flow has an 'initial' step, which is the step the agent starts in when it starts the flow. There are three sample flows in shopify-app/reasoning/app/agent/flows.yaml - one for buying a product from the store, one for submitting an app to be listed in the store, and one initial flow for figuring out what the user wants to do. The agent will move between these flows as necessary. It will abandon the flow it's on and start a new one if you ask it to.

Feel free to tinker with flows.yaml to add your own flows. The format should be self-explanatory, but if you have questions just shoot me a slack!

To demonstrate the capabilities of the state machine, I recorded four sample conversations I had with shopify-app using interactive-test.py. These are in shopify-app/reasoning/app/agent/sample_conversations. Feel free to replicate these yourself. If you can't, or if anything looks weird, please let me know!

TODOs & Cleanup Work

• Slim down the prompts: I kept adding stuff until I got to a point I liked; a lot of it can probably be removed
• Figure out how to pass the state machine to the client via session details: I rigged something that worked but I need to find out if it's the right way to do it
• Build general functionality to trim state machine info in interactive-test.py: the response logging was outputting the entire structure of the state machine - including all flows, states, and transitions - on every response, which made responses very hard to read. I rigged something to redact the state machine from the session variable on output; this can probably be made configurable if we want a general way to say "this variable is huge so don't output it please" (chris - what do you think?)
• remove various debug cruft (mostly pdb imports)

Next Steps

• Remember history when transitioning out of a flow and allow the user to return to where they were if they want
• Auto-gen initial flow so it doesn't have to be specified in flows.yaml
• More testing with more complicated state graphs
• Individual objectives for each transition
• Anything else I'm forgetting!

[mprast]

Extra notes: the agent appears to get "stuck" after applying guardrails a single time. That is to say, if you try to do something unrelated to the current state and the agent stops you, you can no longer switch between flows - the agent will stop you every time. I think it's getting too hung up on the conversation history.

I think the cleanest way around this may be to have a separate graph node that intercepts the output of the RespondToUser node and replaces it if the latest question and answer are unrelated to the objective of the current state machine node. We'd explicitly not include the rest of the conversation here to make sure the agent doesn't get thrown off.

@chrislott what do you think?

[mprast]

also - I think it's probably worth having a separate node type for the initial 'query' flow (the one that describes the options to the user and asks the user what they want to do). as it stands modeling this as its own flow seems to confuse the agent with its vagueness; the agent uses it to circumvent the state guardrails a lot