dottxt-ai / outlines

Structured Text Generation
https://dottxt-ai.github.io/outlines/
Apache License 2.0
9.57k stars 490 forks source link

feat request: Batch inference with outlines applying the same regex #1234

Open alonsosilvaallende opened 3 weeks ago

alonsosilvaallende commented 3 weeks ago

I'm trying to make batch inference with outlines when I'm applying the same regex to a whole dataset. I noticed the following comment in the logits processor: https://github.com/dottxt-ai/outlines/blob/dc31b9b4027e71bd69f213bd3ab7d98b2f0a1529/outlines/processors/base_logits_processor.py#L47-L48 Why is it the case? Is it some hard constraint or can it be relaxed when applying the same regex?