search

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://docs.vllm.ai
Apache License 2.0
23.75k stars 3.41k forks source link

[Feature]: Make `outlines` dependency optional #3794

Open saattrupdan opened 3 months ago

saattrupdan commented 3 months ago

🚀 The feature, motivation and pitch

I'm using a newer version of outlines than v0.0.34, and my application needs the fixes implemented in newer versions of that package. It would be great if vllm could make its structured generation dependencies optional, as I'm not using the internal vllm structured generation, but rather using the logits processors from outlines directly.

Alternatives

No response

Additional context

This prevents me from updating to vllm==0.4.0.

simon-mo commented 3 months ago

Alternatively we can relax the outlines version to not to pin it or set it >=v0.0.34. That will be an even easier fix at the moment :D.

Once we have multiple guided decoding backends I think the packages can be optional.

saattrupdan commented 3 months ago

Alternatively we can relax the outlines version to not to pin it or set it >=v0.0.34. That will be an even easier fix at the moment :D.

That could work! However, note that some of the imports from outlines (RegexFSM for instance) is currently deprecated and will be removed in June, causing ImportErrors in vLLM if nothing changes. But it would at least work until then :)

A quick hotfix could maybe be to relax the versioning and move the outlines imports in guided_logits_processors.py into the classes, to avoid problems if you aren't using that module?

simon-mo commented 3 months ago

The update ticket: https://github.com/vllm-project/vllm/issues/3715