json: cannot unmarshal object into Go struct field ChatRequest.messages of type string

[DGoettlich]

Hi Lennart,

thanks for the library. I'm trying to use it to enforce a json format in a classification task on Ollama models:

messages = [ {"role": "system", "content": sys_prompt}, {"role": "user", "content": prompt} ]

    response = client.chat_completion(
        model=model,
        pydantic_model=validation.ModelResponse,
        messages=messages, 
        format="json",
        options=setting
    )

which often works, but regularly returns such errors:

response = self.ollama_client.chat( model=model, ...<4 lines>... keep_alive=keep_alive ) File "...\ollama_client.py", line 236, in chat
return self._request_stream(

      'POST',
      ^^^^^^^
    ...<10 lines>...
      stream=stream,
      ^^^^^^^^^^^^^^
    )
    ^
  File "...\ollama\_client.py", line 99, in _request_stream
    return self._stream(*args, **kwargs) if stream else self._request(*args, **kwargs).json()
                                                        ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "...\ollama\_client.py", line 75, in _request     
    raise ResponseError(e.response.text, e.response.status_code) from None
ollama._types.ResponseError: json: cannot unmarshal object into Go struct field ChatRequest.messages of type string. 

Any experience with how to solve this?

Thanks in advance!

[lennartpollvogt]

Hey Daniel,

thank you for creating this issue! I was having something in the early days of the development of this library. The Ollama server expects to get a single string (e.g. JSON). I assume I missed considering that somewhere in the processing.

As this seems not to occur very often, it will maybe not easy to reproduce.

Could you tell me which model you are using? And maybe the pydantic model or a similar one?

I hope I will have some time on the weekend.

Best, Lennart

[DGoettlich]

Hi Lennart,

thanks for the quick response!

Models I've tried it with: qwen2.5 and llama3.1. It occurs with both but more frequently with qwen2.5. Its also fairly frequent, actually: happens in around 25% of calls for me.

The pydantic model:

class ModelResponse(BaseModel): """ Validate response """ classification: int = Field(..., description="Indicates if the text contains passages relevant to the question (0 or 1)") confidence: float = Field(..., description="Confidence level of this classification (0 to 1)")

@field_validator('classification')
def validate_relevant(cls, value):
    if isinstance(value, str):
        raise ValueError('classification must be an integer, not a string')
    if value not in (0, 1):
        raise ValueError('classification must be 0 or 1')
    return value

@field_validator('confidence')
def validate_confidence(cls, value):
    if isinstance(value, str):
        raise ValueError('confidence must be a float, not a string')
    if not (0 <= value <= 1):
        raise ValueError('confidence must be between 0 and 1')
    return value

def to_dict(self):
    return self.model_dump()

Let me know if you need more details or can't reproduce the problem, happy to provide more details.

Thanks in advance!

Best, Daniel

[lennartpollvogt]

Hey Daniel,

I found the issue and it seems like I can fix it. 🥳

I will prepare a new version during the weekend.

Btw. could I use the idea for question-answer classification for my docs?

[lennartpollvogt]

Hey Daniel,

the new release is on PyPI and here on GitHub. Would be great to get feedback if it fixed your issue ☺️

Best, Lennart

[DGoettlich]

Hi Lennart,

wonderful, thank you for getting on it so quickly. Seems to have fixed it for me. Of course you can use it for the docs!

Best, Daniel