Structured Outputs: Properties with union types always generate as null unless enum is provided

[jowsey]

When prompting the local server with Structured Outputs and providing a property with any union type (regardless of the inclusion of null), unless an enum is provided, the value will always generate as null, even if it would not make sense for the model to do so:

Prompt:

JSON.stringify({
    model: "example/example",
    messages: [
        {
            role: "system",
            content: "You are a helpful weather assistant.",
        },
        {
            role: "user",
            content: "What is the weather in Seattle?"
        }
    ],
    response_format: {
        type: "json_schema",
        json_schema: {
            name: "get_weather", // Schema loosely copied from the OpenAI docs
            description: "Fetches the weather in the given location",
            strict: true,
            schema: {
                type: "object",
                properties: {
                    location: {
                        type: ["string", "null"], // <-- Change to "string" to fix
                        description: "The location to get the weather for",
                        enum: ["Seattle", "New York"] // <-- Comment this line to always generate null
                    },
                },
                additionalProperties: false,
                required: ["location"]
            }
        }
    }
}),

Actual output:

"message": {
  "role": "assistant",
  "content": "{\"location\": null}"
},

Output when enum is defined, or type is "string":

"message": {
  "role": "assistant",
  "content": "{ \"location\": \"Seattle\"}"
},

Also, looking through the docs, and having not used OpenAI's platform before: LM Studio expects a schema object, whereas OpenAI (whose format LM Studio claims to follow), expects parameters. Not sure if that's an oversight or intended.

Tested with: Gemma 2 9B/27B, Mistral Nemo 13B, and Phi 3.5 mini (all on reasonable quants) LM Studio 0.3.2 AMD Radeon RX 7900 XTX ROCm llama.cpp (Windows) v1.1.9 Windows 11

[muzzah]

Does LM Studio server support the JSON Schema ability? I dont see anything in the docs and Id like to know this as well. Would it be possible to get a clarification by the developers as to what is exactly supported? I see the release blog post says it does but I cannot see anywhere in LM Studio that support providing a json schema.