Chat completion inconsistent (Ollama)

[jamilabreu]

Running chat_completion on Ollama sometimes works, but mostly returns a "can't be blank" error

    messages = [
      %{role: "user", content: "Who were the first three president of the United States?"}
    ]

    Instructor.chat_completion(
      [
        messages: messages,
        mode: :json,
        model: "llama3",
        response_model: %{content: :string}
      ],
      api_key: "ollama",
      api_url: "http://localhost:11434",
      http_options: [receive_timeout: 60000]
    )

{:error,
 #Ecto.Changeset<
   action: nil,
   changes: %{},
   errors: [content: {"can't be blank", [validation: :required]}],
   data: %{},
   valid?: false
 >}

[TwistingTwists]

this might be because llama3 returns the "" empty string as response.

[TwistingTwists]

a usual way to handle this is to retry, either via Instructor.chat_completion(.., retry: 3) , or pattern matching on {:error and making the recursive api call n times