[BUG] json_schema: prefixItems keyword is not constraining output types

[cikkle]

Describe the bug When using the JSON schema feature, the prefixItems keyword is not limiting the possible types of output from the model.

To Reproduce This is an example schema I tested:

 { 
     "type": "array",
     "prefixItems": [
         { "const": "FIXED" },
         { "type": "number" },
         { "type": "string" }     
     ],
     "minItems": 3,
     "maxItems": 3
}

On different attempts, I get back things like:

[12185635797, 56, "Hello"] [20020715, 13, 2002] ["hello", 13, World"] [1.14, 1.13,1.12]

Expected behavior The output should follow the constraints of the prefixItems keyword in the schema, e.g.:

[ "FIXED", 3245.34, "hello"]

Logs Console output shows nothing unusual.

System info (Bugs without this information will go lower on our priority list!) Ubuntu 22.04.4 LTS Python 3.10.12 ROCm 6.1.1

[DocShotgun]

This probably needs more digging to figure out if it's a bug in the underlying library https://github.com/noamgat/lm-format-enforcer.

TabbyAPI doesn't actually do much here with the schema besides passing it to LMFE.

[turboderp]

Are you supplying a $schema key as part of the schema? See here.

[cikkle]

Are you supplying a $schema key as part of the schema? See here.

I'd left it out here but adding "$schema": "https://json-schema.org/draft/2020-12/schema" doesn't appear to change anything.

[turboderp]

It's entirely up to lmfe to parse the schema and constrain the token selection. There seems to be a related issue here, concluded by "prefixItems is works".. so, that's a little ambiguous. Either way it's an upstream issue and there isn't much that can be done about it on this end.

[cikkle]

I sort of figured as much but I thought I'd start here. I've opened an issue with lmfe.