[ERROR][Issue]: {"title":"'messages' array must only contain objects with a 'content' field that is not empty"}[Bug]

[Ansaar-Syed]

Describe the issue

I am using autogen on VSCode (not autogen studio), and an open-sourced LLM (Mistral-7B) on LLM Studio. I keep encountering the error: openai.BadRequestError: Error code: 400 - {'error': "'messages' array must only contain objects with a 'content' field that is not empty."} I noticed that someone raised a similar issue around February 7th, and the suggested fix was to set default_auto_reply="TERMINATE", but this hasn't resolved the issue for me. Below is the code for my UserProxyAgent: `user_proxy = autogen.UserProxyAgent( name="Admin", human_input_mode="NEVER", is_termination_msg=lambda msg: "TERMINATE" in msg.get("content"), max_consecutive_auto_reply=5, code_execution_config={"use_docker": False}, system_message="A human admin. Interact with the planner to discuss the plan. Plan execution needs to be approved " "by this admin.", default_auto_reply="TERMINATE",

llm_config=phi2,

) `

Steps to reproduce

No response

Screenshots and logs

Additional Information

No response

[ManuInNZ]

same here with any models served via LM Studio

[ghost]

Didn't do this either

[scruffynerf]

I have a fix for this, in a 'fix messages queue' function to solve a handful of problems like this. It depends STRONGLY on the model, or rather the model's template. Some fail on any role other than system, user or assistant. Some fail on multiple messages from the same role (ie assistant, user, user is no go), and some like the above, fail due to empty message content.

[Ansaar-Syed]

I have a fix for this, in a 'fix messages queue' function to solve a handful of problems like this. It depends STRONGLY on the model, or rather the model's template. Some fail on any role other than system, user or assistant. Some fail on multiple messages from the same role (ie assistant, user, user is no go), and some like the above, fail due to empty message content.

[Ansaar-Syed]

I have a fix for this, in a 'fix messages queue' function to solve a handful of problems like this. It depends STRONGLY on the model, or rather the model's template. Some fail on any role other than system, user or assistant. Some fail on multiple messages from the same role (ie assistant, user, user is no go), and some like the above, fail due to empty message content.

So what is the fix?

[Ansaar-Syed]

tagging @Hk669 , please look into this.

[Hk669]

cc @scruffynerf @marklysze

[marklysze]

Hey @Ansaar-Syed, this is typical of a situation where the last message in the chat is a role='assistant' message and the LLM thinks it should continue the conversation by predicting the next token. However, because that last assistant message is, essentially, complete, it thinks it should return nothing.

I am working on a message transform that will allow you to enforce a last message that has a user role and a default message, typically something like "Please continue." or just "continue". I haven't created the PR yet but will update this when I have.

So, unfortunately, there isn't a quick fix on this issue, but if you wanted to enforce that last user message you could update client.py and instead of this (I think around line 300):

            params = params.copy()
            params["stream"] = False
            response = completions.create(**params)

you could try adding a couple of lines of code like this:

            params = params.copy()
            params["stream"] = False

            if(params["messages"][-1]["role"] != "user")
                        params["messages"].append({"content": "Please continue.", "role": "user"})

            response = completions.create(**params)

[Ansaar-Syed]

Thank you @marklysze , ill try to do that, also something what i observed is, i was just trying different versions of autogen, 0.2.26 gave me the least issues, idk why, today i tried i didn't get that error, sometimes i do sometimes i don't on 0.2.26 but still i am going to follow your steps.

[marklysze]

Thank you @marklysze , ill try to do that, also something what i observed is, i was just trying different versions of autogen, 0.2.26 gave me the least issues, idk why, today i tried i didn't get that error, sometimes i do sometimes i don't on 0.2.26 but still i am going to follow your steps.

Okay, I'm not sure of the differences in versions... At that point in the code you should also be able to see all messages in params["messages"], it's worth having a look at that and seeing what your messages look like. That may help you work out why your LLM isn't returning a good value.

Also with Mistral 7B, are you using V0.3 of the model?

[Ansaar-Syed]

Thank you @marklysze , ill try to do that, also something what i observed is, i was just trying different versions of autogen, 0.2.26 gave me the least issues, idk why, today i tried i didn't get that error, sometimes i do sometimes i don't on 0.2.26 but still i am going to follow your steps.

Okay, I'm not sure of the differences in versions... At that point in the code you should also be able to see all messages in params["messages"], it's worth having a look at that and seeing what your messages look like. That may help you work out why your LLM isn't returning a good value.

Also with Mistral 7B, are you using V0.3 of the model?

Thank you @marklysze , and i am using V0.1( precisely TheBloke/Mistral-7B-Instruct-v0.1-GGUF), also could you explain what do you mean by seeing the messages in params["messages] like do you want me to print params["messages"] and see what the messages look like?

[marklysze]

Thanks @Ansaar-Syed, I'd strongly suggest trying V0.3, it should be available in LM Studio. V0.3 is better at instruction handling than V0.1/V0.2.

For params["messages"], it's just handy to see what messages are going out to the LiteLLM, that may help you understand why it's responding a certain way. Just remember that the name field on each message does not get used, and that's helpful to understand that the model may not know who has said what. As mentioned, I'm working a message transform class that can help inject the name into each content field of each message so that the model knows which agent is saying what.

[frankiwilson]

you could try adding a couple of lines of code like this:

            params = params.copy()
            params["stream"] = False

            if(params["messages"][-1]["role"] != "user")
                        params["messages"].append({"content": "Please continue.", "role": "user"})

            response = completions.create(**params)

Small syntax correction and then this fixed the issue for me, though it also led to some very long-running conversations between agents:

            params = params.copy()
            params["stream"] = False

            # TEMP FIX to prevent blank content being returned
            if params["messages"][-1]["role"] != "user":
                params["messages"].append({"content": "Please continue.", "role": "user"})

            response = completions.create(**params)```