Amalgam of INVALID JSON issues

[p-i-]

Duplicates

• [X] I have searched the existing issues

Summary 💡

Here's a community challenge!

https://github.com/Torantulino/Auto-GPT/labels/invalid_json

We have 19 PRs pertaining to INVALID JSON

Forever grateful if relevant contributors can get together and figure out a solution for this.

I've created a Discord thread: https://discord.com/channels/1092243196446249134/1095817829405704305/1096547270217977916

EDIT: D'oh that link doesn't quite do the job. In the #dev-autogpt channel, you should see an "Invalid JSON thread" thread.

If we can use the Discord thread for transient convo, and this discussion thread for intel/actionables, we can maybe keep it clean.

Examples 🌈

No response

Motivation 🔦

No response

[carlosplanchon]

I did some work on this front two months ago:

"Code to parse the deepest object on a malformed JSON." It uses ijson for tokenizing, and it follows the interpreter pattern. https://gist.github.com/carlosplanchon/d099001cad31e5596156cefd66b40472

Hope this helps

[DonnyDamon]

The server is empty or i don't have permissions

[waynehamadi]

here is a link to the server: https://discord.gg/autogpt

then you can click on the link given by pi

[waynehamadi]

I am currently looking into this

[GabrielBarberini]

Disclaimer: Nvmd my previous comment, I was hallucinating 😂 . I think now I might have something useful though:

Some thoughts on that issue

Input json string

my_json = """{
    "thoughts":
    {
        "text": "I think I should use the Google Search command to search for information on the internet.",
        "reasoning": "The Agent is designed to help me with basic tasks, and searching for information on the internet is one of them. Using the Google Search command will allow me to find information quickly and efficiently.",
        "plan": "- Use the Google Search command to search for information on the internet.\n- Once the search results are displayed, I will read them and decide on the next course of action.",
        "criticism": "I must ensure that I don't rely on the GPT Agent too much, and constantly work to develop my own abilities.",
        "speak": "I am going to use the Google Search command to search for information on the internet."
    },
    "command": {
        "name": 'google',
        "args":{
            "input": "what is the capital of France?"
        }
    }
}"""

fix_and_parse_json output

json.decoder.JSONDecodeError: Invalid control character at: line 6 column 92 (char 452)

line 6 column 92 => "\n-" OBS: Sometimes I get \\n- and it breaks just like for the case above. OBS2: Also, sometimes get_command(response) gets non-json_string structures, e,g:

I apologize for the previous errors, let me try again to provide the response.

{
    "thoughts": {
        "text": "I can use the 'google' command to search for resources on automated code testing. After conducting an initial search, I can save any relevant information to a file using the 'write_to_file' command. I will then use the 'browse_website' command to find a useful resource on automated code testing.",
        "reasoning": "Automated code testing is an essential procedure to ensure that the code functions correctly. By searching for resources on automated code testing, I can improve my knowledge on this topic and create more efficient code.",
        "plan": "- Use the 'google' command to search for resources on automated code testing\n- Save relevant information to files using the 'write_to_file' command\n- Use the 'browse_website' command to find useful resources on automated code testing",
        "criticism": "I should ensure that I use the information to become better at automated code testing and other related concepts. I should also ensure that I minimize the number of files I create.",
        "speak": "I will use the 'google' command to search for resources on automated code testing."
    },
    "command": {
        "name": "google",
        "args": {
            "input": "Automated code testing resources"
        }
    }
}

which leads to json.decoder.JSONDecodeError in json_parser.py

OBS3: Another option I noticed is when receiving an almost-valid json like:

"command": {
        "name": "google",
        "args": {
            "input": "Automated code testing resources"
        }
    }

(missing { } around) and json_utils.correct_json is not solving it as well.

OBS4: Kind of a mix between OBS 3 and OBS 2, here the JSON is almost valid (missing enclosing {}) and there is trash in the end

assistant_reply

"do_nothing", args: "" I will wait for 20 seconds and try the browse_website command again since there is a rate limit issue.

[GabrielBarberini]

I did something that solved the issue for me. Feel free to make any amends or ignore 🤷‍♂️😝

/scripts/json_parser.py

change line 29 to 51 to this:

def fix_and_parse_json(    
    json_str: str,
    try_to_fix_with_gpt: bool = True
) -> Union[str, Dict[Any, Any]]:
    """Fix and parse JSON string"""
    try:
        # remove invalid text and get only JSON string
        start_index = json_str.find('{')
        end_index = json_str.rfind('}')
        json_str = json_str[start_index:end_index+1]
        json_str = json_str.replace('\t', '')
        json_str = json_str.replace('\n', '')
        json_str = json_str.replace('\\n', '')
        return json.loads(json_str)
    except json.JSONDecodeError as _:  # noqa: F841
        json_str = correct_json(json_str)
        try:
            return json.loads(json_str)
        except json.JSONDecodeError as _:  # noqa: F841
            try: 
                return json.loads("{"+json_str+"}")
            except json.JSONDecodeError as _:  # noqa: F841 
                pass

It solves the issue for 3 of the scenarios I shared above:

1. Removes \n and \\n from json strings.
2. Extract only the json from json_strings that sometimes comes with trash.
3. As a last attempt, it adds { } to the json string.

It is definitely not collectively exhaustive, but as I said, I didn't get into the loop after that running for a few hours.

[philliplastcall]

I think this issue bleeds over to file related commands on windows as well : Error: [WinError 123] The filename, directory name, or volume label syntax is incorrect: \'C:\\ai\\Auto-GPT\\auto_gpt_workspace\\\' \nHuman Feedback: GENERATE NEXT COMMAND JSON '] json loads error Invalid \escape: line 5 column 170 (char 978) json loads error - fix invalid escape Invalid \escape: line 5 column 175 (char 983)

[waynehamadi]

To anyone reading this, do you still experience the AI spitting out things that are not JSON ?

I have mostly seen people complain about the fact that now the AI spits out a json but the wrong command.

[GodotH]

FIXED (maybe)

I had this issue while trying to create files on my disk (Win11) I fixed it by simply using Terminal with Admin rights, running the exact same prompts, it worked flawlessly.

Could be risky I know, but it works.

[phadeb]

FIXED (maybe)

I had this issue while trying to create files on my disk (Win11) I fixed it by simply using Terminal with Admin rights, running the exact same prompts, it worked flawlessly.

Could be risky I know, but it works.

Always been running as admin but got the problem

[esd100]

still having this problem...what is the fix?

[brianberneker]

I'm not sure if this has been proposed yet, but at least giving it a graceful way to fail and break out of the loop might be helpful. Also, having the ability to save progress so that a user can terminate and resume from where they left off might be a way to "reset" the bug.

Also, deep re-parsing of the bad json or treating a missing key with special handling other than a generic error and retry could help. A prompt restating the required format has probably already been tried :(

[rskonieczka]

Missing 'command' object in JSON

The JSON object is invalid. THOUGHTS: None REASONING: None CRITICISM: None NEXT ACTION: COMMAND = Error: ARGUMENTS = Missing 'command' object in JSON SYSTEM: Command Error: threw the following error: Missing 'command' object in JSON Traceback (most recent call last): File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/home/swami/Pobrane/autogpt/Auto-GPT/autogpt/__main__.py", line 5, in <module> autogpt.cli.main() File "/home/swami/.local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__ return self.main(*args, **kwargs) File "/home/swami/.local/lib/python3.10/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/home/swami/.local/lib/python3.10/site-packages/click/core.py", line 1635, in invoke rv = super().invoke(ctx) File "/home/swami/.local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/swami/.local/lib/python3.10/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/home/swami/.local/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func return f(get_current_context(), *args, **kwargs) File "/home/swami/Pobrane/autogpt/Auto-GPT/autogpt/cli.py", line 90, in main run_auto_gpt( File "/home/swami/Pobrane/autogpt/Auto-GPT/autogpt/main.py", line 186, in run_auto_gpt agent.start_interaction_loop() File "/home/swami/Pobrane/autogpt/Auto-GPT/autogpt/agent/agent.py", line 112, in start_interaction_loop assistant_reply = chat_with_ai( File "/home/swami/Pobrane/autogpt/Auto-GPT/autogpt/llm/chat.py", line 165, in chat_with_ai agent.summary_memory = update_running_summary( File "/home/swami/Pobrane/autogpt/Auto-GPT/autogpt/memory_management/summary_memory.py", line 78, in update_running_summary content_dict = json.loads(event["content"]) File "/usr/lib/python3.10/json/__init__.py", line 346, in loads return _default_decoder.decode(s) File "/usr/lib/python3.10/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[VectorZhao]

Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "F:\Github\Auto-GPT\autogpt__main.py", line 5, in autogpt.cli.main() File "C:\Users\zy\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1130, in call return self.main(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\zy\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1055, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "C:\Users\zy\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1635, in invoke rv = super().invoke(ctx) ^^^^^^^^^^^^^^^^^^^ File "C:\Users\zy\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\zy\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 760, in invoke return callback(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\zy\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\decorators.py", line 26, in new_func File "F:\Github\Auto-GPT\autogpt\cli.py", line 90, in main run_auto_gpt( File "F:\Github\Auto-GPT\autogpt\main.py", line 186, in run_auto_gpt return f(get_current_context(), args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\Github\Auto-GPT\autogpt\cli.py", line 90, in main run_auto_gpt( File "F:\Github\Auto-GPT\autogpt\main.py", line 186, in run_auto_gpt agent.start_interaction_loop() File "F:\Github\Auto-GPT\autogpt\agent\agent.py", line 113, in start_interaction_loop assistant_reply = chat_with_ai( ^^^^^^^^^^^^^ File "F:\Github\Auto-GPT\autogpt\llm\chat.py", line 165, in chat_with_ai agent.summary_memory = update_running_summary( ^^^^^^^^^^^^^^^^^^^^^^^ File "F:\Github\Auto-GPT\autogpt\memory_management\summary_memory.py", line 78, in update_running_summary content_dict = json.loads(event["content"]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\zy\AppData\Local\Programs\Python\Python311\Lib\json__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\zy\AppData\Local\Programs\Python\Python311\Lib\json\decoder.py", line 340, in decode raise JSONDecodeError("Extra data", s, end) json.decoder.JSONDecodeError: Extra data: line 17 column 1 (char 791)

[lc0rp]

Related to https://github.com/Significant-Gravitas/Auto-GPT/issues/21 (Invalid Json)

[simwai]

I did some work on this front two months ago:

"Code to parse the deepest object on a malformed JSON." It uses ijson for tokenizing, and it follows the interpreter pattern. https://gist.github.com/carlosplanchon/d099001cad31e5596156cefd66b40472

Hope this helps

How to use this?

[github-actions[bot]]

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

[simwai]

I had an idea to solve this wrong JSON format problem. Maybe you could handle it by just sending the wrong formatted JSON string to ChatGPT and specify that it needs to output a correct validated JSON without any additional text. This is how I usually handle such problems..

[carlosplanchon]

@simwai It may work, yes, but when you have a lot of queries trying to return structured JSON that becomes very costly really fast.

[simwai]

That's true. Maybe, it would be more optimal to add the required structure in the prompt which asked for the JSON. It could also be enough to forbid ChatGPT in the prompt to write invalid JSON. Most optimal is I guess to do a normal fix approach (without ChatGPT) when a wrong JSON appears, and when the validation of the fixed JSON fails to use the fix by additional ChatGPT query as last rescue.

[github-actions[bot]]

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

[jamesk9526]

Im still having these issues. any fixes for it yet?

[mgunnin]

Having this issue as well.

[simwai]

@Pwuts thanks for the fix!!!