More modular interface?

[simon-mo]

Is it possible to have some sort of interface that's not lm-format-enforcer wrapping vLLM, rather, users can directly install both library and configure it using SamplingParams?

The pseudocode in my mind:

SamplingParams(
    logits_processors=[lmformatenforcer.JSONSchema(MyPydanticClass)],
    max_tokens=16,
)

Here's a similar proposal to Outlines: https://github.com/outlines-dev/outlines/issues/163#issuecomment-1820069503

[noamgat]

the logits processor needs the tokenizer to prepare itself. To make an API like this possible, the logitsprocessor interface would have to be more than a Callable, but a class with some init(tokenizer) functionality. This is inconsistent with how other inference libraries open logits processing APIs, which is why I chose this approach with vLLM.

Its technically possible, but would require changes to vLLM's SamplingParams, and I actually think the current interface is the right one.