MrFCow
commented
5 months ago I believe it's because the Azure LLM API call not processing the Message bace into string when passing to request
checking the code on Azure: https://github.com/cpacker/MemGPT/blob/da21e7edbc43067e3c317a726f831b1b6c5628e9/memgpt/llm_api/azure_openai.py
the function azure_openai_chat_completions_request take in the data as a dictionary, and then directly pass it to requests.post, when I debug the data, the dictionary is like:
{messages: [<memgpt.data_types.Message object at 0x7fc19b57eba0>, <memgpt.data_types.Message object at 0x7fc19b57e690>,,,], 'toots': ...}
so we can see unless we convert the Message object in the messages key to what Azure/openAI expected format, the request library would give error in the JSON serialization.
On the otherhand, the Azure LLM api supporting class seems implemented differently compare to Open AI, Anthropic and Cohere (while Google one is another different implementation)
According to function signature, OpenAI, Anthropic and Cohere would accept request data as ChatCompleteionRequest in their request method, while Google and Azure is using a dictionary as data input.
one of the potential function could be converting the Message object in the data["messages"] into json string using the Message class to_openai_dict() like:
data["messages"] = [x.to_openai_dict() for x in data["messages"]]
I think that would at least allow Azure to accept the result and response...
My final comment is, Azure OpenAI handling is like a step child in most library I worked with regarding LLM, the code are mostly not as polish as Open AI, so finger cross and best wish to us Azure user.
anujkapoorcg
ljhskyso
I installed memgpt version 0.3.12 on Mac using pip3 install pymemgpt
I am using Azure OpenAI LLM and I have all the settings configured: export AZURE_OPENAI_KEY=xxxxx export AZURE_OPENAI_ENDPOINT=xxxxxx export AZURE_OPENAI_DEPLOYMENT=xxxxxx export AZURE_OPENAI_VERSION=2024-02-01
here is how I configured it anujkapoor@Anujs-Macbook-Pro Python % memgpt configure ? Select LLM inference provider: azure ? Select default model (recommended: gpt-4): gpt-3.5-turbo-16k ? Select embedding provider: local ? Select storage backend for archival data: postgres ? Enter postgres connection string (e.g. postgresql+pg8000://{user}:{password}@{ip}:5432/{database}):postgresql+pg8000 ://memgpt:memgpt@localhost:5432/memgpt ? Select storage backend for recall data: postgres ? Enter postgres connection string (e.g. postgresql+pg8000://{user}:{password}@{ip}:5432/{database}): postgresql+pg8000://memgpt:memgpt@localhost:5432/memgpt
Here is the output of memgpt run: anujkapoor@Anujs-Macbook-Pro Python % memgpt run
? Would you like to select an existing agent? Yes ? Select agent: AmusingXylophonist
🔁 Using existing agent AmusingXylophonist
Hit enter to begin (will request first MemGPT message)
An exception occurred when running agent.step(): Traceback (most recent call last): File "/Users/anujkapoor/PycharmProjects/azure-openai/Labfiles/02-azure-openai-api/Python/MemGPT/memgpt/main.py", line 395, in run_agent_loop new_messages, user_message, skip_next_user_input = process_agent_step(user_message, no_verify) File "/Users/anujkapoor/PycharmProjects/azure-openai/Labfiles/02-azure-openai-api/Python/MemGPT/memgpt/main.py", line 364, in process_agent_step new_messages, heartbeat_request, function_failed, token_warning, tokens_accumulated = memgpt_agent.step( File "/Users/anujkapoor/PycharmProjects/azure-openai/Labfiles/02-azure-openai-api/Python/MemGPT/memgpt/agent.py", line 804, in step raise e File "/Users/anujkapoor/PycharmProjects/azure-openai/Labfiles/02-azure-openai-api/Python/MemGPT/memgpt/agent.py", line 719, in step response = self._get_ai_reply( File "/Users/anujkapoor/PycharmProjects/azure-openai/Labfiles/02-azure-openai-api/Python/MemGPT/memgpt/agent.py", line 437, in _get_ai_reply raise e File "/Users/anujkapoor/PycharmProjects/azure-openai/Labfiles/02-azure-openai-api/Python/MemGPT/memgpt/agent.py", line 414, in _get_ai_reply response = create( File "/Users/anujkapoor/PycharmProjects/azure-openai/Labfiles/02-azure-openai-api/Python/MemGPT/memgpt/llm_api/llm_api_tools.py", line 115, in wrapper raise e File "/Users/anujkapoor/PycharmProjects/azure-openai/Labfiles/02-azure-openai-api/Python/MemGPT/memgpt/llm_api/llm_api_tools.py", line 88, in wrapper return func(*args, kwargs) File "/Users/anujkapoor/PycharmProjects/azure-openai/Labfiles/02-azure-openai-api/Python/MemGPT/memgpt/llm_api/llm_api_tools.py", line 215, in create return azure_openai_chat_completions_request( File "/Users/anujkapoor/PycharmProjects/azure-openai/Labfiles/02-azure-openai-api/Python/MemGPT/memgpt/llm_api/azure_openai.py", line 121, in azure_openai_chat_completions_request raise e File "/Users/anujkapoor/PycharmProjects/azure-openai/Labfiles/02-azure-openai-api/Python/MemGPT/memgpt/llm_api/azure_openai.py", line 100, in azure_openai_chat_completions_request response = requests.post(url, headers=headers, json=data) File "/Users/anujkapoor/Library/Python/3.10/lib/python/site-packages/requests/api.py", line 115, in post return request("post", url, data=data, json=json, kwargs) File "/Users/anujkapoor/Library/Python/3.10/lib/python/site-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, kwargs) File "/Users/anujkapoor/Library/Python/3.10/lib/python/site-packages/requests/sessions.py", line 575, in request prep = self.prepare_request(req) File "/Users/anujkapoor/Library/Python/3.10/lib/python/site-packages/requests/sessions.py", line 486, in prepare_request p.prepare( File "/Users/anujkapoor/Library/Python/3.10/lib/python/site-packages/requests/models.py", line 371, in prepare self.prepare_body(data, files, json) File "/Users/anujkapoor/Library/Python/3.10/lib/python/site-packages/requests/models.py", line 511, in prepare_body body = complexjson.dumps(json, allow_nan=False) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/init.py", line 238, in dumps kw).encode(obj) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 199, in encode chunks = self.iterencode(o, _one_shot=True) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 257, in iterencode return _iterencode(o, 0) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 179, in default raise TypeError(f'Object of type {o.class.name} ' TypeError: Object of type Message is not JSON serializable ? Retry agent.step()? (Y/n)
Screenshots I tried to debug and this is what is going in the post call to Azure:
Here is the value of data {'messages': [<memgpt.data_types.Message object at 0x1c8ecbeb0>, <memgpt.data_types.Message object at 0x1672340a0>, <memgpt.data_types.Message object at 0x167234220>, <memgpt.data_types.Message object at 0x167253520>, <memgpt.data_types.Message object at 0x167253580>], 'tools': [{...}, {...}, {...}, {...}, {...}, {...}, {...}, {...}], 'tool_choice': 'auto', 'user': '00000000-0000-0000-0000-000000000000'} special variables: class: <class 'dict'> class_getitem: <built-in method class_getitem of type object at 0x11071cb10> contains: <built-in method contains of dict object at 0x1c8ed24c0> delattr: <method-wrapper 'delattr' of dict object at 0x1c8ed24c0> delitem: <method-wrapper 'delitem' of dict object at 0x1c8ed24c0> dir: <built-in method dir of dict object at 0x1c8ed24c0> doc: "dict() -> new empty dictionary\ndict(mapping) -> new dictionary initialized from a mapping object's\n (key, value) pairs\ndict(iterable) -> new dictionary initialized as if via:\n d = {}\n for k, v in iterable:\n d[k] = v\ndict(**kwargs) -> new dictionary initialized with the name=value pairs\n in the keyword argument list. For example: dict(one=1, two=2)" eq: <method-wrapper 'eq' of dict object at 0x1c8ed24c0> format: <built-in method format of dict object at 0x1c8ed24c0> ge: <method-wrapper 'ge' of dict object at 0x1c8ed24c0> getattribute: <method-wrapper 'getattribute' of dict object at 0x1c8ed24c0> getitem: <built-in method getitem of dict object at 0x1c8ed24c0> gt: <method-wrapper 'gt' of dict object at 0x1c8ed24c0> hash: None init: <method-wrapper 'init' of dict object at 0x1c8ed24c0> init_subclass: <built-in method init_subclass of type object at 0x11071cb10> ior: <method-wrapper 'ior' of dict object at 0x1c8ed24c0> iter: <method-wrapper 'iter' of dict object at 0x1c8ed24c0> le: <method-wrapper 'le' of dict object at 0x1c8ed24c0> len: <method-wrapper 'len' of dict object at 0x1c8ed24c0> lt: <method-wrapper 'lt' of dict object at 0x1c8ed24c0> ne: <method-wrapper 'ne' of dict object at 0x1c8ed24c0> new: <built-in method new of type object at 0x11071cb10> or: <method-wrapper 'or' of dict object at 0x1c8ed24c0> reduce: <built-in method reduce of dict object at 0x1c8ed24c0> reduce_ex: <built-in method reduce_ex of dict object at 0x1c8ed24c0> repr: <method-wrapper 'repr' of dict object at 0x1c8ed24c0> reversed: <built-in method reversed of dict object at 0x1c8ed24c0> ror: <method-wrapper 'ror' of dict object at 0x1c8ed24c0> setattr: <method-wrapper 'setattr' of dict object at 0x1c8ed24c0> setitem: <method-wrapper 'setitem' of dict object at 0x1c8ed24c0> sizeof: <built-in method sizeof of dict object at 0x1c8ed24c0> str: <method-wrapper 'str' of dict object at 0x1c8ed24c0> subclasshook: <built-in method subclasshook of type object at 0x11071cb10> function variables: 'messages': [<memgpt.data_types.Message object at 0x1c8ecbeb0>, <memgpt.data_types.Message object at 0x1672340a0>, <memgpt.data_types.Message object at 0x167234220>, <memgpt.data_types.Message object at 0x167253520>, <memgpt.data_types.Message object at 0x167253580>] special variables: function variables: 0: <memgpt.data_types.Message object at 0x1c8ecbeb0> 1: <memgpt.data_types.Message object at 0x1672340a0> 2: <memgpt.data_types.Message object at 0x167234220> 3: <memgpt.data_types.Message object at 0x167253520> 4: <memgpt.data_types.Message object at 0x167253580> len(): 5 'tools': [{'type': 'function', 'function': {...}}, {'type': 'function', 'function': {...}}, {'type': 'function', 'function': {...}}, {'type': 'function', 'function': {...}}, {'type': 'function', 'function': {...}}, {'type': 'function', 'function': {...}}, {'type': 'function', 'function': {...}}, {'type': 'function', 'function': {...}}] 'tool_choice': 'auto' 'user': '00000000-0000-0000-0000-000000000000' len(): 4
I guess, because of object being passed in messages field, it is not able to serialize this as JSON
MemGPT Config [defaults] preset = memgpt_chat persona = sam_pov human = basic
[model] model = gpt-3.5-turbo-16k model_endpoint = https://openai-xxxxxxx.openai.azure.com/ model_endpoint_type = azure context_window = 16385
[embedding] embedding_endpoint_type = local embedding_model = BAAI/bge-small-en-v1.5 embedding_dim = 384 embedding_chunk_size = 300
[archival_storage] type = postgres path = /Users/anujkapoor/.memgpt/chroma uri = postgresql+pg8000://memgpt:memgpt@localhost:5432/memgpt
[recall_storage] type = postgres path = /Users/anujkapoor/.memgpt uri = postgresql+pg8000://memgpt:memgpt@localhost:5432/memgpt
[metadata_storage] type = postgres path = /Users/anujkapoor/.memgpt uri = postgresql+pg8000://memgpt:memgpt@localhost:5432/memgpt
[version] memgpt_version = 0.3.12
[client] anon_clientid = 00000000-0000-0000-0000-000000000000