search

noamgat / lm-format-enforcer

Enforce the output format (JSON Schema, Regex etc) of a language model
MIT License
1.42k stars 65 forks source link

What is the difference wrt LMQL? #3

Closed ogencoglu closed 11 months ago

ogencoglu commented 11 months ago

What is the difference of your library vs LMQL?

noamgat commented 11 months ago

LMQL mixes two approaches to controlling output - prompt engineering and constraints.

In its constraint page you can see what can be done with constraints (that prevent the LLM from generating tokens that violate the format). Note that JSON Schema is not there.

JSON Schema is done via prompt engineering and verification, similar to Typechat.

This library only uses the constraint approach, and is able to correctly filter the tokens allowed at the next timestep that can eventually turn into correct output, ensuring format-compliant outputs in the first iteration.

ogencoglu commented 11 months ago

Thanks for the swift reply!