gpt-3.5-turbo-1106 issues

[MiskoH]

Hey. I ran into a problem, that I started to get error like this:

File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\pydantic\main.py", line 755, in __getattr__ raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}') from exc AttributeError: 'ChatCompletionMessage' object has no attribute 'lower'

while trying to use TeachableAgent. I also got similar error:

File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\autogen\agentchat\groupchat.py", line 211, in _mentioned_agents count = len(re.findall(regex, " " + message_content + " ")) # Pad the message to help with matching TypeError: can only concatenate str (not "ChatCompletionMessage") to str

when trying to use automatic selection of next agent in group chat.

I have noticed, that these errors started to appear after I changed my model from gpt-3.5-turbo to gpt-3.5-turbo-1106. Also, in two weeks the gpt-3.5-turbo will point to gpt-3.5-turbo-1106 so the issue would definitely appear later.

I tried to find the problem and I came to conclusion, that the gpt-3.5-turbo-1106 needs more specific messages. While I didn't find a solution for the first error yet, the solution for the second one is pretty easy.

The problem is in groupchat.py file on line 163 - all I had to do was to change the line from this: "content": f"Read the above conversation. Then select the next role from {[agent.name for agent in agents]} to play. Only return the role.",

to this:

"content": f"Read the above conversation. Then select the next role from this list:{[agent.name for agent in agents]} to play. Only return the role.",

I assume there will be more places to fix, definitely after the gpt-3.5-turbo will start to point to gpt-3.5-turbo-1106.

Also complete traces: First:

Traceback (most recent call last): File "\Chat.py", line 10, in <module> groupChat.start_chat() File "\GroupChatManager.py", line 41, in start_chat user_proxy.initiate_chat( File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\autogen\agentchat\conversable_agent.py", line 550, in initiate_chat self.send(self.generate_init_message(**context), recipient, silent=silent) File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\autogen\agentchat\conversable_agent.py", line 348, in send recipient.receive(message, self, request_reply, silent) File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\autogen\agentchat\conversable_agent.py", line 481, in receive reply = self.generate_reply(messages=self.chat_messages[sender], sender=sender) File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\autogen\agentchat\conversable_agent.py", line 906, in generate_reply final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"]) File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\autogen\agentchat\groupchat.py", line 271, in run_chat reply = speaker.generate_reply(sender=self) File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\autogen\agentchat\conversable_agent.py", line 906, in generate_reply final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"]) File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\autogen\agentchat\contrib\teachable_agent.py", line 122, in _generate_teachable_assistant_reply new_user_text = self.consider_memo_retrieval(user_text) File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\autogen\agentchat\contrib\teachable_agent.py", line 203, in consider_memo_retrieval if "yes" in response.lower(): File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\pydantic\main.py", line 755, in __getattr__ raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}') from exc AttributeError: 'ChatCompletionMessage' object has no attribute 'lower'

Second:

Traceback (most recent call last): File "\Chat.py", line 10, in <module> groupChat.start_chat() File "\GroupChatManager.py", line 41, in start_chat user_proxy.initiate_chat( File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\autogen\agentchat\conversable_agent.py", line 550, in initiate_chat self.send(self.generate_init_message(**context), recipient, silent=silent) File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\autogen\agentchat\conversable_agent.py", line 348, in send recipient.receive(message, self, request_reply, silent) File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\autogen\agentchat\conversable_agent.py", line 481, in receive reply = self.generate_reply(messages=self.chat_messages[sender], sender=sender) File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\autogen\agentchat\conversable_agent.py", line 906, in generate_reply final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"]) File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\autogen\agentchat\groupchat.py", line 269, in run_chat speaker = groupchat.select_speaker(speaker, self) File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\autogen\agentchat\groupchat.py", line 172, in select_speaker mentions = self._mentioned_agents(name, agents) File "\AppData\Local\anaconda3\envs\new_test_env\lib\site-packages\autogen\agentchat\groupchat.py", line 211, in _mentioned_agents count = len(re.findall(regex, " " + message_content + " ")) # Pad the message to help with matching TypeError: can only concatenate str (not "ChatCompletionMessage") to str

[afourney]

Thanks. This is likely related to #765. A quick fix was pushed yesterday but it just prevents the crash. I will invest some time today to see if I can work out why this is happening with the preview models.

[victordibia]

Might we consider this addressed?

[afourney]

@victordibia I still don't have a root cause for the ChatCompletionMessage making it that far down into the call-stack.

[sonichi]

Is #791 related?

[afourney]

Yes. I believe so.

Again, if there are steps to reproducing this, (e.g., using a particular version of the OpenAI library), I'd love to get in there and trace it. I've not been able to reproduce this on my end yet. I'll try more today

[sonichi]

openai==1.3.5 I believe will cause it.

[sonichi]

@bytecod3r @kevin666aa have been working on it I think.

[yiranwu0]

Yes, #791 will make sure that return from generate_oai_reply is a dict or str, which should solve this issue.

[yiranwu0]

This issue should be solved with the latest updates.