Give the general agent access to a 'marketplace of tools'

[evangriffiths]

At the moment, the range of actions our general agent can take is limited by the tools/Functions registered with the agent before it is run.

Using the mech-client is a step in the direction of giving the agent access to new functionality at runtime. However, it's not currently used at such, due to the following missing features:

• ability to search/list available tools
• ability to query the client to get a tool's description
• ability to query the client to get a tool's input and output schema

(Another useful but optional feature would be for tools to have a variable cost - currently they are hardcoded at $0.01/call).

Initial discussions needed to determine if these features can be added to mech-client, and if we can collaborate with them on this.

[gabrielfior]

Some points for further discussion:

1. It's possible to retrieve metadata from all Gnosis mechs, by a. Calling function mapServices(service_id) from the ServiceRegistry and fetching metadata from IPFS using ("f01701220" + <result_from_mapService.configHash). b. The logic from a. needs to be double-checked, but there must be a way to obtain a dictionary of service_id -> metadata (which can serve as tool description).

2. Non-blockchain tools

   • Let's consider getMarkets as a tool - we simply query the subgraph. So this could either be a. a centralized backend serving data via endpoints or b. a mech answering to requests.

3. Blockchain tools

   • Let's consider placeBet as a tool - what needs to occur is that a signed blockchain tx gets sent to a validator's mempool (i.e. somebody calls web3py.eth.send_raw_transaction after web3.eth.account.sign_transaction was called by the owner of the funds)
   • It's hard to imagine the caller handing over the private keys to the mech. It's less hard to imagine it assigning the required conditions, like erc20 approvals, etc, but still not very elegant.
   • One possible way would be for the mech to build the transaction data (basically, call prepare_tx from PMAT) and return data to the caller agent, who then can sign & submit the tx.
   • This way, we could have arbitrary transactions (i.e. functions) being constructed for the agent.
   • Only problem with that is that the tx data becomes opaque, i.e. you cannot directly decode the data argument of the built tx_params. To solve this, the mech would have to reply with the ABI of the contract being called, so that the caller could verify which method was called with which arguments.
   • Finally, in this way, mechs could keep getting paid a flat value, since gas fees are paid by the caller when submitting the tx.

In my head, this involves the following necessary (but not sufficient) steps:

• Get metadata from existing mechs (that, afaik, only fulfill LLM prompts)
• Add new mechs (1 per function) that responds with dictionaries of built transactions (see below) produced by prepare_tx

  {'value': 0, 'gas': 46025, 'maxFeePerGas': 5160000007, 'maxPriorityFeePerGas': 5159999993, 'chainId': 100, 'from': '0xe91D153E0b41518A2Ce8Dd3D7944Fa863463a97d', 'nonce': 1, 'to': '0xe91D153E0b41518A2Ce8Dd3D7944Fa863463a97d', 'data': '0x095ea7b3000000000000000000000000e91d153e0b41518a2ce8dd3d7944fa863463a97d0000000000000000000000000000000000000000000000000000000000000001'}

• Refactor mech-client to be able to fetch both LLM prompt responses and dictionaries of built transactions and act upon these (i.e. sign & send built transactions or simply continue execution having the response of an LLM or a quantity fetched by a subgraph)

Very happy to discuss more about this.

[kongzii]

Before anything else, we can add a dummy class into our code that will simulate such a marketplace, and the agent should use it (and save it to his state) to get what functions are available. THen he can periodically call it to check for new ones.

[gabrielfior]

I wrote a simple script (see commits in the linked branch) for fetching description from a given service (from the IPFS hash associated to the service, e.g. for serviceId 6 this IPFS hash contains its description.

[gabrielfior]

Sharing some more references here: -> This mech (https://github.com/valory-xyz/mech/blob/main/packages/valory/customs/prepare_tx/prepare_tx.py) can be used for preparing txs and returning the txParams, which can then be signed & submitted by the agent. This would allow for an arbitrary number of functions for the agents to call, all of them deployed as mechs.

[gabrielfior]

To be implemented -> 1 mech that is able to build txParams for buyTokens -> A MechLoader object that fetches all mechs and returns descriptions about them and how to call them -> Further Github issue to create more mechs

[evangriffiths]

For the

ability to query the client to get a tool's input schema subtask, this either requires:

• mech tools to have a schema (open ai calls it function specification) stored in the ipfs metadata alongside the description (e.g. here https://gateway.autonolas.tech/ipfs/f0170122025d9324b3229021fec87d68ee5727a21b822f273b730244835b1d9dd5bc10aa8). If this is the case, we should use the openai spec syntax -- see https://cookbook.openai.com/examples/how_to_call_functions_with_chat_models#basic-concepts -- as we'd have to covert to that anyway
• have a way of auto-generating this from the code. This should be theoretically possible if we can programatically get the mech tool's code from ipfs, and the code is documented.

For the second option, I thought you could get the code from "code_uri":"ipfs://bafybeidn3pgtofyom7mwusjm73ydy2librqcofrxwqhxajyh2rrdvlyngy", but I can't see any of these IPFS links working. Will ask on Olas Telegram.

[evangriffiths]

from notion notes yesterday's meeting:

How to give caller of Mechs their function signature?

1. open-autonomy push to update ipfs contents for each mech tool to contain function signature
2. hack the description of the mech to contain the args in a specific format

Define a schema for function signatures. Pick openai (==langchain) tool schema

[gabrielfior]

Added some notes for the Olas metadata feature here -> https://gnosischain.notion.site/Olas-Marketplace-of-tools-600faf9c49c943e9afcd6e3cdedfd0f3

[gabrielfior]

Another option for having a marketplace of tools with metadata for each tool (in a GraphQL-like schema) would be to use ComposeDB (they presented during DappCon24). It's also built on top of libp2p (like IPFS) but allows for schemas, which would be helpful when publishing new tools.