search

DevXT-LLC / ezlocalai

ezlocalai is an easy to set up local artificial intelligence server with OpenAI Style Endpoints.
MIT License
66 stars 12 forks source link

Internal error when sending more than 1 message #3

Closed mja00 closed 7 months ago

mja00 commented 7 months ago

I'm attempting to get this to work but seem to be running into issues. When I request a completion using a single message in the array, messages = [{"role": "system", "content": "Hello, Markov!"},] for example, I get a reply. The instant I attempt to add another message, for instance something like {"role": "system", "content": "You are an AI named Markov that exists in a Discord server called The Igloo."} I then get error 500s with the following error in the log

  File "/app/app.py", line 91, in chat_completions
    return LLM(**c.model_dump()).chat(messages=c.messages)
  File "/app/local_llm/__init__.py", line 340, in chat
    prompt = f"\nASSISTANT's RULE: {message.content}"
AttributeError: 'dict' object has no attribute 'content'

It seems like the payload isn't being parsed properly.

Josh-XT commented 7 months ago

Hello and thank you! I just updated the code and released v0.0.30 to fix the issue. I saw your PR after I already committed the update, thank you for that!