Invalid JSON error related to non-JSON strings

[aphexddb]

Duplicates

• [X] I have searched the existing issues

Steps to reproduce 🕹

1. Modified attempt_to_fix_json_by_finding_outermost_brackets in scripts/main.py and added the following line:

   logger.typewriter_log("[raw value of json_string]", json_string)

2. Started in debug mode and asked "In two sentences, summarize the current state of the Snow Crab ecosystem" under the goal of researcher, the model returns an empty JSON string. This is reproducible.

Current behavior 😯

Debug output:

Input: In two sentences, summarize the current state of the Snow Crab ecosystem
SYSTEM:  Human feedback: In two sentences, summarize the current state of the Snow Crab ecosystem
  Token limit: 4000
  Memory Stats: {'index_name': 'auto-gpt', 'index_options': [], 'index_definition': [b'key_type', b'HASH', b'prefixes', [b'auto-gpt:'], b'default_score', b'1'], 'attributes': [[b'identifier', b'data', b'attribute', b'data', b'type', b'TEXT', b'WEIGHT', b'1'], [b'identifier', b'embedding', b'attribute', b'embedding', b'type', b'VECTOR']], 'num_docs': '1', 'max_doc_id': '1', 'num_terms': '58', 'num_records': '59', 'inverted_sz_mb': '0.00035190582275390625', 'vector_index_sz_mb': '6.1705780029296875', 'total_inverted_index_blocks': '58', 'offset_vectors_sz_mb': '7.534027099609375e-05', 'doc_table_size_mb': '7.2479248046875e-05', 'sortable_values_size_mb': '0', 'key_table_size_mb': '2.765655517578125e-05', 'records_per_doc_avg': '59', 'bytes_per_record_avg': '6.254237174987793', 'offsets_per_term_avg': '1.3389830589294434', 'offset_bits_per_record_avg': '8', 'hash_indexing_failures': '0', 'total_indexing_time': '0.184', 'indexing': '0', 'percent_indexed': '1', 'number_of_uses': 3, 'gc_stats': [b'bytes_collected', b'0', b'total_ms_run', b'0', b'total_cycles', b'0', b'average_cycle_time_ms', b'-nan', b'last_run_time_ms', b'0', b'gc_numeric_trees_missed', b'0', b'gc_blocks_denied', b'0'], 'cursor_stats': [b'global_idle', 0, b'global_total', 0, b'index_capacity', 128, b'index_total', 0], 'dialect_stats': [b'dialect_1', 0, b'dialect_2', 1, b'dialect_3', 0]}
  Token limit: 4000
  Send Token Count: 1223
  Tokens remaining for response: 2777
  ------------ CONTEXT SENT TO AI ---------------
  System: The current time and date is Fri Apr 14 09:39:25 2023

  System: This reminds you of these events from your past:
['Assistant Reply: {\n    "thoughts": {\n        "text": "I will wait for the user\'s input to determine what to do next.",\n        "reasoning": "Since my actions depend on user input, I will wait for their command before taking any action.",\n        "plan": "- Wait for user input.",\n        "criticism": "",\n        "speak": "How may I assist you?"\n    },\n    "command": {\n        "name": "do_nothing",\n        "args": {}\n    }\n} \nResult: Human feedback:  In two sentences, summarize the current state of the Snow Crab ecosystem \nHuman Feedback:  In two sentences, summarize the current state of the Snow Crab ecosystem ']

  User: Determine which next command to use, and respond using the format specified above:

  Assistant: {
    "thoughts": {
        "text": "I will wait for the user's input to determine what to do next.",
        "reasoning": "Since my actions depend on user input, I will wait for their command before taking any action.",
        "plan": "- Wait for user input.",
        "criticism": "",
        "speak": "How may I assist you?"
    },
    "command": {
        "name": "do_nothing",
        "args": {}
    }
}

  System: Human feedback:  In two sentences, summarize the current state of the Snow Crab ecosystem

  User:  In two sentences, summarize the current state of the Snow Crab ecosystem

  ----------- END OF CONTEXT ----------------
json There has been a decline in the population of snow crab in recent years due to several reasons such as climate change and overfishing. Several conservation measures have been implemented to prevent further decline and promote the recovery of the population.
json loads error Expecting value: line 1 column 1 (char 0)
  Warning: Failed to parse AI output, attempting to fix.
 If you see this warning frequently, it's likely that your prompt is confusing the AI. Try changing it up slightly.
  ------------ JSON FIX ATTEMPT ---------------
  Original JSON: ``json
There has been a decline in the population of snow crab in recent years due to several reasons such as climate change and overfishing. Several conservation measures have been implemented to prevent further decline and promote the recovery of the population.
``

Fixed JSON: {
  "command": {
      "name": "",
      "args":{}
  },
  "thoughts":
  {
      "text": "There has been a decline in the population of snow crab in recent years due to several reasons such as climate change and overfishing. Several conservation measures have been implemented to prevent further decline and promote the recovery of the population.",
      "reasoning": "",
      "plan": "-",
      "criticism": "",
      "speak": "There has been a decline in the population of snow crab in recent years due to several reasons such as climate change and overfishing. Several conservation measures have been implemented to prevent further decline and promote the recovery of the population."
  }
}
END OF FIX ATTEMPT

EA-GPT THOUGHTS:  There has been a decline in the population of snow crab in recent years due to several reasons such as climate change and overfishing. Several conservation measures have been implemented to prevent further decline and promote the recovery of the population.
REASONING:
PLAN:
-
CRITICISM:
Attempting to fix JSON by finding outermost brackets
There has been a decline in the population of snow crab in recent years due to several reasons such as climate change and overfishing. Several conservation measures have been implemented to prevent further decline and promote the recovery of the population.[raw value of json_string]
Error: Invalid JSON, setting it to empty JSON now.

Expected behavior 🤔

Model responses should be in JSON format that the program can read. The debug log shows vua [raw value of json_string] that an emptry string was passed to attempt_to_fix_json_by_finding_outermost_brackets.

Your prompt 📝

ai settings

ai_goals:
- Be the best exective assistant possible
- View calendar information and schedule meetings when requested
- Answer general questions
- Play to your strengths as a Large Language Model
ai_name: EA-GPT
ai_role: An AI designed to be exective assistant

prompt

In two sentences, summarize the current state of the Snow Crab ecosystem

[Drlordbasil]

awaiting fix for this too, spending hella $ on trying :P

[Drlordbasil]

cant wait till peeps become millionaires and make the creator a millionaire from this too.

[p-i-]

Community effort to sort out this JSON fail once and for all at https://github.com/Torantulino/Auto-GPT/issues/1407

We have 17 open PRs for fixing this, and the merge-team are stacked.

[z-x-x136]

logger.typewriter_log("[raw value of json_string]", json_string)，What is stored in the json_string variable?

[aphexddb]

logger.typewriter_log("[raw value of json_string]", json_string)，What is stored in the json_string variable?

@z-x-x136 This is logging line that was added, and it appears json_string was an empty string passed to the attempt_to_fix_json_by_finding_outermost_brackets method.

def attempt_to_fix_json_by_finding_outermost_brackets(json_string):
    if cfg.speak_mode and cfg.debug_mode:
      speak.say_text("I have received an invalid JSON response from the OpenAI API. Trying to fix it now.")
    logger.typewriter_log("Attempting to fix JSON by finding outermost brackets\n")
    logger.typewriter_log("[raw value of json_string]", json_string)
    ...

[z-x-x136]

def attempt_to_fix_json_by_finding_outermost_brackets(json_string):
    if cfg.speak_mode and cfg.debug_mode:
      speak.say_text("I have received an invalid JSON response from the OpenAI API. Trying to fix it now.")
    logger.typewriter_log("Attempting to fix JSON by finding outermost brackets\n")
    logger.typewriter_log("[raw value of json_string]", json_string)

Thanks

[Pwuts]

Closing as one of many duplicates; superseded by #1407