noamgat
commented
2 months ago I see two approaches possible:
Easy approach, with a lot of preprocessing in the start if your string is long:
1 - Collect all n^2 substrings of your source string (you probably want to filter duplicates, or use prefix tree methods) 2 - Construct a UnionParser from StringParser of each string
"Lazy computation" approach: no preprocessing, but slower per_token speeds, and more coding required:
You would have to write your own character level parser, that would work something like this:
- in the init, it receives the text in which it quotes
- it keeps a "parsed_string" state of type string, and simply adds to it in "add_character" (self.parsed_string += new_character)
- in get_allowed_characters(), it scans the source string for parsed_string, and collects the following characters of all occurences. These are the allowed characters().
- can_end() always return True.
This should work. I'm not sure about performance as get_allowed_characters() scans your entire source string, and depending on its length, it could be costly. But it will return correct result.
There might be more computationally efficient methods using prefix trees.
accupham
iiLaurens
I work in a field where the consequences of hallucinations are large. The mere possibility of a hallucination makes most LLM projects inviable. It would be of great help if I can provide some sort of guarantee that whatever the model produces is purely extracted from the prompt. That is, there should be a direct hit when searching for that string in the prompt.
Imagine a schema where I ask a question, and I want to collect reasoning, an answer and a quote. The answer would be limited to Yes/No/Unknown, which is easy to enforce. Then the reasoning can be free format, because I want to expose the reasoning of the model (which increases accuracy). Then for the quote, I want the model to only allow sample substrings from the prompt. This last part I am struggling with. Do you think something like this would be possible with LM-format-enforcer?