dottxt-ai / outlines

Structured Text Generation

https://dottxt-ai.github.io/outlines/

Apache License 2.0

9.71k stars 497 forks source link

Minor Changes on Top of Isamus EXL2 PR (Docs, Enable FSM/CFG, Fix Tokenizer) #1191

Closed lapp0 closed 1 month ago

lapp0 commented 1 month ago

@isamu-isozaki did a great job implementing a fully functioning outlines.models.exllamav2. This PR introduces minor fixes and a documentation update.

This PR rebases with main and squashes his commits into one: https://github.com/dottxt-ai/outlines/commit/d1f5609b2ac25b47aa01ffbe98d4d2f70317475c

Changes

All changes noted in core functionality PR: https://github.com/dottxt-ai/outlines/pull/1010

New

Automatically download necessary models for tests to run successfully
Make OutlinesExLlamaV2Tokenizer, which is compatible with cfg
For models.exllamav2 enable generate.cfg and generate.fsm
Document how to install third party ExLlamaV2 with logits processor functionality

Closes

closes https://github.com/dottxt-ai/outlines/pull/1010 (original PR)
Fixes https://github.com/dottxt-ai/outlines/issues/1009
Fixes https://github.com/dottxt-ai/outlines/issues/807