search

noamgat / lm-format-enforcer

Enforce the output format (JSON Schema, Regex etc) of a language model
MIT License
1.01k stars 46 forks source link

Added optional banned tokens parameter to vllm processor. #60

Closed kmccleary3301 closed 4 months ago

kmccleary3301 commented 5 months ago

Wanted to stop model from outputting consecutive newlines when using JSON schema. As well as \r and \t characters.

noamgat commented 5 months ago

Hi! I understand what you're trying to do, but I don't think modifying LMFE this is the right way to approach this. 1) This is vLLM only 2) Since vLLM's LogitsProcessor API supports multiple logits processors, it is just as possible to create a brand new logits processor that does only what you are doing here, instead of modifying the LMFE logits processor. I think that it is a much cleaner way to go.