Using LLM Agents to Complete Simulation Agent Action

[DarioPanada]

Hello,

I would like to implement a scenario where a simulation agent action is complete by accessing an external service. For example, if an agent's attempted action was:

'Alex searches on CupcakeHub for strawberry cupcake recipes.'

I would like to trigger a call from code to the CupcakeHub API* to retrieve said recipe, and then feed it back to the agent as an observation. Eg:

'Alex finds on CupcakeHub that the recipe for the strawberry cupcake recipe involves 100g of flour, 2 eggs...'

Is this possible?

My current thoughts:

1. The most suitable place would be the update_from_player method of the GameMaster class;
2. An LLM would be used to infer if the player action matched the use-case to trigger the external service, and if so to produce the request payload. That would be achieved via a prompt similar to:

Given the following attempted action <action-attempt>{action_attempt}</action-attempt>, output an answer in the following format

<evaluation>YES if the player wants to access CupcakeHub, NO otherwise</evaluation>
<explanation>An explanation of how you reached said conclusion</explanation>
<query>Query to submit to cupcake hub</query>

3. Based on that, a conditional logic which, if the evaluation is YES, calls the CupcakeHub API via its client, obtains the response and feeds it back to the agent. (This could be achieved perhaps using LangChain agents, which allow integrations of LLM with external services https://www.promptingguide.ai/research/llm-agents#llm-agent-applications)

I am wondering if my reasoning is correct or if you think this could be better implemented by 'hooking into' other components of the simulation life-cycle?

*Made up service, but you get the idea

[jzleibo]

Yep, I think that's a great way to implement that. It's very similar to the way we have done this kind of thing ourselves.