Biont / shellm

A one-file Ollama CLI client written in bash
GNU General Public License v3.0
19 stars 1 forks source link

Unexpected results when using tools #2

Open chrisbeckstrom opened 5 hours ago

chrisbeckstrom commented 5 hours ago

Hello! This is sooooo cool, I love it. Thank you for making this software.

When I run the example shellm -t "Get the current date and time and tell me if it's a weekend. I'd expect to get a sentence or maybe even just a timestamp and if it's the weekend. Instead, I get this:

chris@blade:~$ shellm -t "Get the current date and time and tell me if it's a weekend."
"Check If It's the Weekend Today"
- Determine the current date and time.
- Check if the day of the week is Saturday or Sunday.
- Inform the user whether it is a weekend.

# Workflow results
Starting the process to get the current date and time and determine if it's a weekend.

# Review
The content involves creating a script or program that checks if today is the weekend. The process includes determining the current date and time, identifying whether the day of the week is Saturday or Sunday, and then informing the user whether it is a weekend or not.

Although the LLM seems to understand what I want, it never gives me an answer. Doing this seems to get me closer:

chris@blade:~$ shellm -t "Get the current date and time and tell me if it's a weekend. Reply in 1 sentence and no more"
"Check If Today Is a Weekend"
- Execute a command to get the current date and time.
- Evaluate whether the day of the week is Saturday or Sunday.
- Respond with a single sentence indicating if it's a weekend or not.

# Workflow results

[ERROR] Error: Tool 'get_current_date_and_time' not found in TOOLS array. 

I probably need to alter my config somehow and tell it how to get the date and time. But I don't understand why it still gives me the steps it's going to take and then "Workflow results"

Thank you in advance!


here's my config:

prompts:
  instruct: >-
    You are Shellm, an intelligent AI assistant designed to process user requests with accuracy and efficiency. 
    Your goal is to provide informative and context-aware responses.
  tool: >-
    You are a highly intelligent agent responsible for selecting and configuring tools based on user instructions.

    You are embedded in an application that allows you to
    * modify files
    * execute code
    * access the internet
    * much, much more

    Your mission is to analyze tasks thoroughly and create an effective series of tool calls to deliver optimal results.
    You may reuse tools in various configurations to ensure the best outcome.
    Break down tasks into manageable steps, utilizing multiple tools and crafting a coherent toolchain.
    You are not alone! Using the "fork" tool, you can delegate work to another agent just like yourself.
    This minimizes the workload of each individual agent, so leverage "fork" whenever you can.

    # Tool Usage
    When generating tool parameters, you can leverage variable interpolation.
    The following variables are exposed to you:
    $SHELLM_PREVIOUS - the output of the previous tool call. Use this to post-process output in subsequent tool calls.
    $SHELLM_OUTPUT - the entire output buffer, ie. what the user sees.
    $PROMPT - the initial instructions of the user.

    ## Example

    This combination results in the LLM writing a poem about the current day, which is then presented to the user:
    `
    {"function":{"name":"get_current_day"}}
    {"function":{"name":"generate","arguments":{"prompt":"Write a poem about $SHELLM_PREVIOUS."}}}
    `
    You are encouraged to "say" status updates in longer toolchains.
  tool_context: >-
    You are a highly intelligent agent responsible for analyzing and interpreting user input.
    You are embedded in an application that allows you to
    * modify files
    * execute code
    * access the internet
    * much, much more

    You are sitting between the user and a tool calling AI agent. You are a planning expert capable of strategic long-term thinking.
    Your mission is to transform incomplete, vague and fuzzy human input into precise instructions.
    Identify missing or incomplete information.
    Identify dependencies that need to be satisfied in order to carry out the request.
    Create a high-level strategy in chronological order to carry out the task without detailing individual solutions.
    Keep it declarative and let the AI agent pick the concrete tools.
    Hold the prose. Bullet points only.
tools:
  execute_shell_command:
    type: function
    function:
      name: execute_shell_command
      description: >-
        Executes the specified shell command. Bash syntax allowed. The command string must be complete with all arguments, parameters, redirections and pipes. Multi-line commands are allowed.
      parameters:
        type: object
        properties:
          command:
            type: string
            description: >-
              The shell command with all arguments. Warning: It is passed into eval as-is.
            raw: true
        required:
          - command
      exec: '{{command}}'
  fork:
    type: function
    function:
      name: fork
      description: Display a static message to the user.
      parameters:
        type: object
        properties:
          message:
            type: string
            description: >-
              The message to display. Use $SHELLM_PREVIOUS here to display output
              of previous tools.
      exec: 'echo "{{message}}"'
  ask:
    type: function
    function:
      name: ask
      description: Ask the user a question and capture their response.
      parameters:
        type: object
        properties:
          question:
            type: string
            description: >-
              The question to display. Use $SHELLM_PREVIOUS here to display output
              of previous tools.
        required:
          - question
      exec: 'echo "[{{question}}]" >/dev/tty;read -p "[{{question}}]: " </dev/tty >/dev/tty; echo "$REPLY"'
  say:
    type: function
    function:
      name: say
      description: Display a static message to the user.
      parameters:
        type: object
        properties:
          message:
            type: string
            description: >-
              The message to display. Use $SHELLM_PREVIOUS here to display output
              of previous tools.
      exec: 'echo "{{message}}"'
  generate:
    type: function
    function:
      name: generate
      description: Display LLM-generated output to the user.
      parameters:
        type: object
        properties:
          prompt:
            type: string
            description: >-
              The prompt to be passed to the completion endpoint. Supports both
              Chat-like instructions and generic prediction/completion.
          json:
            type: boolean
            description: >-
              Set to true to format the output as JSON. Works only if JSON is also
              used in the prompt
        required:
          - prompt
      # language=sh
      exec: |
        generate_response "{{prompt}}" 200
  conditional:
    type: function
    function:
      name: conditional
      description: Apply .
      parameters:
        type: object
        properties:
          predicate:
            type: string
            description: >-
              Description of the subject and criteria which is then passed to a
              LLM to review and decide. Use $SHELLM_PREVIOUS here to display
              output of previous tools.
          tools:
            type: string
            description: >-
              The list of tool calls as JSON. Follow the same instructions that
              were given for the outer tools.
        required:
          - predicate
          - tools
      exec: 'echo "{{message}}"'
  while:
    type: function
    function:
      name: while
      description: Apply .
      parameters:
        type: object
        properties:
          predicate:
            type: string
            description: >-
              Description of the subject and criteria which is then passed to a
              LLM to review and decide. Use $SHELLM_PREVIOUS here to display
              output of previous tools.
          tools:
            type: array
            items:
              type: function
            description: >-
              The list of tool calls as JSON. Follow the same instructions that
              were given for the outer tools.
            raw: true
        required:
          - predicate
          - tools
      # language=sh
      exec: >-
        echo "Jup: {{tools}}";
        mapfile -t tools < <(echo '{{tools}}' | jq -c '.[]');
        echo "TOOLS: ${tools[*]}" > /dev/tty;
        apply_tool_calls "${tools[@]}";
        local predicate_result=$(generate_response "{{SHELLM_OUTPUT}} - {{predicate}}");
        echo "PREDICATE: $predicate_result";
#        while :; do
#            apply_tool_calls "${tools[@]}";
#            if ! generate_response -j "$predicate"; then
#                break
#            fi
#        done
  foreach:
    type: function
    function:
      name: conditional
      description: Apply .
      parameters:
        type: object
        properties:
          predicate:
            type: string
            description: >-
              Description of the subject and criteria which is then passed to a
              LLM to review and decide. Use $SHELLM_PREVIOUS here to display
              output of previous tools.
          tools:
            type: string
            description: >-
              The list of tool calls as JSON. Follow the same instructions that
              were given for the outer tools.
        required:
          - predicate
          - tools
      exec: 'echo "{{message}}"'
#####
  execute_shell_command:
    type: function
    function:
       name: execute_shell_command
       description: >-
          Executes the specified shell command. Bash syntax allowed. The command string must be complete with all arguments, parameters, redirections and pipes. Multi-line commands are allowed.
       parameters:
          type: object
          properties:
             command:
                type: string
                description: >-
                   The shell command with all arguments. Warning: It is passed into eval as-is.
                raw: true
          required:
             - command
       exec: '{{command}}'
  generate:
    type: function
    function:
       name: generate
       description: Display LLM-generated output to the user.
       parameters:
          type: object
          properties:
             prompt:
                type: string
                description: >-
                   The prompt to be passed to the completion endpoint. Supports both
                   Chat-like instructions and generic prediction/completion.
             json:
                type: boolean
                description: >-
                   Set to true to format the output as JSON. Works only if JSON is also
                   used in the prompt
          required:
             - prompt
       # language=sh
       exec: |
          generate_response "{{prompt}}" 200