Closed cikkle closed 1 month ago
Hi @cikkle, thanks for the report!
As you can see in the docs the support for external $ref
s hasn't been implemented in the C++ json schema -> grammar converter yet (will need to use CURL, it's on my todo list).
I guess we should probably find a way to issue warnings about it or even hard-fail (currently we just silently degrade to "anything goes" so the items in your example are just any json object), but in the meantime as a workaround you might wanna try and run python examples/json_schema_to_grammar.py schema.json
and pass the resulting grammar to your server call, if possible. Or possibly, paste the schema in your prompt to rely on a mix of good will from the model your items and schema constraint only for the high-level shape of the output.
Cheers
Sorry, I might have a pretty basic misunderstanding as far as the spec, the docs, or the terms used. I took "external" to mean a reference to a schema in another file; the characters
and emotes
in the $def
block in the second example should be all local, right? Even given the notes in the docs I thought that would work.
Anyhow, for my use case I don't have much of a problem with just copying enums around the schema where they need to go, but the fallback behavior did surprise me, so thanks for looking at this.
Sorry, I might have a pretty basic misunderstanding as far as the spec, the docs, or the terms used. I took "external" to mean a reference to a schema in another file; the
characters
andemotes
in the$def
block in the second example should be all local, right? Even given the notes in the docs I thought that would work.
Ohh sorry I completely misread your bug report, thanks for the clarification 🫣
Anyhow, for my use case I don't have much of a problem with just copying enums around the schema where they need to go, but the fallback behavior did surprise me, so thanks for looking at this.
It looks like it's a bug specific to the C++ implementation of the JSON Schema -> Grammar conversion, I'll try to send a fix shortly.
In the meantime you can use python examples/json_schema_to_grammar.py schema.json
which seems to work with your example.
This issue was closed because it has been inactive for 14 days since being marked as stale.
What happened?
I'm using the json_schema feature in llama-server. Using a simple prompt like
Write a dialog between Alice and Biff
, if I send a schema like:I get back an array of responses in the format I'd expect, like:
Things stop working right if I try to put the enums in separate definitions. The following schema:
...gives me arbitrary things like:
The output should follow the same format in both, but I get an object with random properties in place of the enum, and possibly more random things afterward (in this run, it was a bonus object tagging along, but it can vary).
Notably if I reorder the properties to put "dialog" before "character" I'll actually get the
dialog
property and string I asked for, so things only seem to go off the rails when it reaches one of the referenced enums.I'm aware json_schema currently has some known bugs and features yet to implemented, but I didn't see anything in the readme I thought this would fall under. Terminal output from llama-server doesn't appear to show anything relevant but it's included for completeness.
Name and Version
o0@hades:~/ai/llama.cpp$ ./llama-cli --version version: 3203 (b5a5f34e) built with cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 for x86_64-linux-gnu
What operating system are you seeing the problem on?
Linux
Relevant log output