Gemini does not respect the order of the properties in a schema

[seungduk-yanolja]

Description of the bug:

Let's imagine a situation where you want an answer and a rationale from the LLM. You want it to be in a JSON format like

{
  "choice": 4,
  "rationale": "The answer is 4 because..."
}

If the LLM completes the choice property first, then the next rationale can be made up to justify the choice. However, if the rational comes first like

{
  "rationale": "The answer is 4 because...",
  "choice": 4
}

Then, the choice will be determined by rationale. It is similar to CoT.

It is important that the LLM follows the given order of the properties in an object. However, I found that Gemini SDK ignores it.

    properties: MutableMapping[str, "Schema"] = proto.MapField(
        proto.STRING,
        proto.MESSAGE,
        number=3,
        message="Schema",
    )

I think this one better to be a repeated, not a map field, to keep the order of the properties.

Actual vs expected behavior:

Gemini SDK ignores the order of the properties in an object.

Any other information you'd like to share?

No response

[seungduk-yanolja]

Any updates? also there is another issue: https://github.com/google/generative-ai-go/issues/211

[httplups]

I am having the same issue that Gemini ignores the order; it tends to hallucinate

[httplups]

Hi. I was able to make the model follow the order by changing the field names to 'step_one' and 'step_two'. It is not ideal, but its a workaround to have an structured output with reasoning.

[tfriedel]

You can give a propertyOrdering property with a list of the property names, then it will respect that order.

[MarkDaoust]

@httplups The object it returns is unordered. It may just be printed alphabetically, the order it generates them may just be incidental.

propertyOrdering

The API could add something like that to solve the problem.

[IsaSalemHame]

Any update?