search

noamgat / lm-format-enforcer

Enforce the output format (JSON Schema, Regex etc) of a language model
MIT License
1.01k stars 46 forks source link

Limit string length #23

Closed xydisla closed 7 months ago

xydisla commented 7 months ago

I am trying to store plain text in a string output, typically a couple sentences. However, sometimes my model will generate a couple paragraphs, slowing down inference. Is it possible to limit the length of string-type outputs within a JSON? Limiting the length of all output typically results in malformed JSON, since output will be terminated early.

noamgat commented 7 months ago

The just released v0.6.4 respects the min/max string length limitation in JSON Schema. See the unit test to see how to define it in your pydantic model.