Use suffix automata for extractive generation

dottxt-ai / outlines

Structured Text Generation

https://dottxt-ai.github.io/outlines/

Apache License 2.0

9.44k stars 479 forks source link

Use suffix automata for extractive generation #770

Closed rlouf closed 1 month ago

rlouf commented 7 months ago

Discussed in https://github.com/outlines-dev/outlines/discussions/575

^{Originally posted by **JosephGatto** January 24, 2024} Is it possible to restrict the output to be any contiguous span of text from the input? In other words, I am wondering if Outlines can be used to do what would effectively be extractive inference?

xiaoyunwu commented 2 months ago

https://paperswithcode.com/paper/automata-based-constraints-for-language-model it is possible to support this?