Server crashes with unsupported regex characters

Structured generation is currently supported in ml-client through Outlines.

Regex structured generation with unsupported regex characters causes the server to crash.

Here is an example:

import time
messages = [
    SystemMessage(
        content="You are a helpful assistant."
    ),
    HumanMessage(
        content="What are the characteristics of an amoeba?"
    )
]
first_chunk = True
output = ""
for chunk in llm.stream(messages, extra_body={"outlines_type":"regex", "regex": r"\{(?:[^{}\n\r]|(?R))*\}"}):
    if first_chunk: 
        first_chunk = False
    print(chunk.content, end="", flush=True)

Error:

Reset server due to Unknown group-extension: 'R' (Context: '(?R))\}' information. How can I assist you today? Traceback (most recent call last): File "/home/ms5267@drexel.edu/blockentropy/ml-client/llm_exl2_dynamic_gen.py", line 440, in process_prompts filters = [RegexFilter(outlines_dict["regex"], hf_tokenizer)] File "/home/ms5267@drexel.edu/anaconda3/envs/bellm/lib/python3.10/site-packages/outlines/integrations/exllamav2.py", line 127, in init fsm = RegexGuide(regex_string=regex_string, tokenizer=tokenizer) File "/home/ms5267@drexel.edu/anaconda3/envs/bellm/lib/python3.10/site-packages/outlines/fsm/guide.py", line 145, in init ) = create_states_mapping(regex_string, tokenizer) ... ... ... File "/home/ms5267@drexel.edu/anaconda3/envs/bellm/lib/python3.10/site-packages/interegular/patterns.py", line 507, in conc parts.append(self.obj()) File "/home/ms5267@drexel.edu/anaconda3/envs/bellm/lib/python3.10/site-packages/interegular/utils/simple_parser.py", line 38, in w return m(self, args, kwargs) File "/home/ms5267@drexel.edu/anaconda3/envs/bellm/lib/python3.10/site-packages/interegular/patterns.py", line 514, in obj return self.group() File "/home/ms5267@drexel.edu/anaconda3/envs/bellm/lib/python3.10/site-packages/interegular/utils/simple_parser.py", line 38, in w return m(self, *args, *kwargs) File "/home/ms5267@drexel.edu/anaconda3/envs/bellm/lib/python3.10/site-packages/interegular/patterns.py", line 519, in group return self.extension_group() File "/home/ms5267@drexel.edu/anaconda3/envs/bellm/lib/python3.10/site-packages/interegular/utils/simple_parser.py", line 38, in w return m(self, args, kwargs) File "/home/ms5267@drexel.edu/anaconda3/envs/bellm/lib/python3.10/site-packages/interegular/patterns.py", line 582, in extension_group raise InvalidSyntax(

The reason is due to 'R' recursive is not supported by Outlines, this should return the same exception to the client instead of crashing the server.

blockentropy / ml-client

Server crashes with unsupported regex characters #18