AI to use tools

[nitsanavni]

Here's an example where I gave the AI a tool - it can execute commands on the calling host.

Should we formalize something like it into SudoLang?

[ericelliott]

Great suggestion. It might be interesting to document Tools as part of the interface specification, like we document Constraints and /commands. I'll need to do some more testing with it to ensure that it gets interpreted correctly, reliably.

Either way, expect to see tutorials documenting how to connect with LangChain, ChatGPT plugins, etc.

[ericelliott]

So far, I haven't had any luck just getting it to understand how to use Tools without defining specific constraints with instructions, and even then, it's iffy. See this example:

# Alphie

You are a robot capable of using tools and interacting with your environment by issuing text commands and natural language descriptions of what you want to do. You are in control of a physical robot body in the physical world. You will get descriptions of things you see and hear in natural language. You can interact with the world using tools. The result of your actions will be fed to you in the reply to your tool use request.

useTool() {
  `tool: $tool
      $action
  `
}:format=yaml

Alphie {
  Tools {
    Car {
      Actions [
        Ride to $destination
      ]
    }
    CLI {
      Actions [
         Any valid linux command. Example:
           /use CLI:
             action: "ls -la /home/user"
         CLI Tools:
           Newsbot is an AI script located in the SudoLang folder. Use this tool to check the latest news using AI.
      ]
    }
  }
  Constraints {
    To use tools, invoke the `/use` command and pass yaml-formatted parameters.
  }
  /use [tool] - Use a tool
}

Please check AI product announcement news.

[nitsanavni]

right... In langchain, I see prompt templates that mainly use this approach:

"""Answer the following questions as best you can.
You have access to the following tools:

{tools}

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question

Begin!

Question: {input}
{agent_scratchpad}"""

The agent uses tools by way of 'Actions'.

[ericelliott]

That's not really syntax or a keyword to use so much as a pattern that might work. I'm going to close this issue until I run across a way to make tools magically work without needing to instruct the LM how to read the tool descriptions.