Is multiple choice constraints possible via StringParser?

[ivysdad]

Similar to how outlines allows for the LLM to respond with a specific word from a set of choices, can we do the same with LMFE?

I've been researching a lot of the different enforcers and I really liked the setup/process here. Also like, huge kudos for explaining how enforcement works, every other repo kinda just says "hey this works!" Plus the addition of diagnostics is really up my alley, as I like to understand how my prompts are impacting the output.

My particular use case is using LLMs to label data, right now I've been iteratively testing manually, but it seems that an enforcer might be a good fail safe -- especially if I mainly want it to output just a "Yes" or "No" answer. Similar to the example Outlines gave. I am non-technical, sorry if this was as a bad question! Would love to get code examples directly if this is possible thank you!

[AriX]

Yes, you can do this by creating UnionParser with multiple StringParsers! Apologies, I cannot write a code sample at this moment.

[ivysdad]

Yes, you can do this by creating UnionParser with multiple StringParsers! Apologies, I cannot write a code sample at this moment.

okay! Im going to try to look into this w/ copilot and try to get something together! if you do get a chance id love to see an example, thank you so much for taking the time to point me in the right direction!

[ivysdad]

worked w/ copilot to get this working! Was a lot easier (less scary) than I thought it would be! Here is my code in case anyone else wanted a similar solution, here "yes" and "no" were my multiple choice labels. Basically just followed the example up until the json part.

# Import String & Union Parser classes
from lmformatenforcer.characterlevelparser import StringParser, UnionParser

# Create StringParser instances for "Yes" and "No"
yes_parser = StringParser("Yes")
no_parser = StringParser("No")

# Create a UnionParser with the "Yes" and "No" parsers
union_parser = UnionParser([yes_parser, no_parser])

# Create a logits processor with the union_parser
logits_processor = build_vllm_logits_processor(tokenizer_data, union_parser)

# Create a SamplingParams object
sampling_params = SamplingParams()
sampling_params.max_tokens = DEFAULT_MAX_NEW_TOKENS
sampling_params.logits_processors = [logits_processor]

# Generate texts from the prompts
outputs = llm.generate(prompts, sampling_params=sampling_params)

# Load the data
df_training = pd.read_csv(dataset_path)

# Extract the generated text and add it to the dataframe
df_training['LLM_output'] = [output.outputs[0].text for output in outputs]

# Display only the 'content' and 'LLM_output' columns
display(df_training[['content', 'LLM_output']])