search

predibase / lorax

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
https://loraexchange.ai
Apache License 2.0
1.86k stars 125 forks source link

Added eager prefill option #524

Closed tgaddair closed 6 days ago

tgaddair commented 6 days ago

Added to lorax-launcher:

--eager-prefill