From my understanding of the code in TGI (building the FSM and using the FSM to bias logits) and discussions on the hf-dottxt-outlines-collab TGI interacts with Outlines in the following way:
The FSM is compiled in Python using RegexFSM
The instance of RegexFSM is used to bias logits
While (2) can remain in Python we would like to port the logic necessary to build the FSM (1) to Rust. This means porting:
From my understanding of the code in TGI (building the FSM and using the FSM to bias logits) and discussions on the
hf-dottxt-outlines-collab
TGI interacts with Outlines in the following way:RegexFSM
RegexFSM
is used to bias logitsWhile (2) can remain in Python we would like to port the logic necessary to build the FSM (1) to Rust. This means porting:
create_fsm_index_tokenizer
;interegular
library used to build a DFA from a regular expression;