Handling of tool errors (e.g. Database)

[mattlindsey]

Example below shows where a Tool raises an error, but the Assistant doesn't recognize it and happily gives wrong answer. Maybe the Assistant can at least acknowledge that an error happened, and in the specific case below if the Assistant gets access to a 'schema' function/tool, it should work better.

[mattlindsey]

Would just making sure any error message is put on the Thread help? I don't see it anywhere above except the log message.

Python langchain has elaborate configuration for 'Handling Tool Errors,' but is that unnecessary complexity?

[mattlindsey]

This looks better:

 #<Langchain::Message:0x0000000105a35ab0
  @content=
   "PG::UndefinedColumn: ERROR:  column \"name\" does not exist\nLINE 1: SELECT COUNT(*) FROM users WHERE LENGTH(name) > 5\n                                                ^\n",
  @role="tool",
  @tool_call_id="call_yKmQDXW3OpJM28m8JO7IowAy",
  @tool_calls=[]>,
 #<Langchain::Message:0x0000000105beed98
  @content=
   "I apologize for the confusion. It seems that the column \"name\" does not exist in the users table. Could you please provide the correct column name for the user's name?",
  @role="assistant",
  @tool_call_id=nil,
  @tool_calls=[]>]
#<Langchain::Message:0x0000000103eeab48 @content="Actually I meant the first_name column.", @role="user", @tool_call_id=nil, @tool_calls=[]>,
 #<Langchain::Message:0x000000010593f070
  @content="",
  @role="assistant",
  @tool_call_id=nil,
  @tool_calls=
   [{"id"=>"call_99rBiKnqGwnWC2B6Q708lmHE",
     "type"=>"function",
     "function"=>{"name"=>"database", "arguments"=>"{\n  \"input\": \"SELECT COUNT(*) FROM users WHERE LENGTH(first_name) > 5\"\n}"}}]>,
 #<Langchain::Message:0x000000010593eda0 @content="[{:count=>66}]", @role="tool", @tool_call_id="call_99rBiKnqGwnWC2B6Q708lmHE", @tool_calls=[]>,
 #<Langchain::Message:0x0000000105936ce0
  @content="There are 66 users in the users table whose first_name has a length greater than 5.",
  @role="assistant",
  @tool_call_id=nil,
  @tool_calls=[]>]

Return error below in database.rb. Maybe in other tools too?

    #
    # Evaluates a sql expression
    #
    # @param input [String] sql expression
    # @return [Array] results
    #
    def execute(input:)
      Langchain.logger.info("Executing \"#{input}\"", for: self.class)

      db[input].to_a
    rescue Sequel::DatabaseError => e
      Langchain.logger.error(e.message, for: self.class)
      e.message     ###### ADDED THIS #######
    end