search

bentoml / BentoVLLM

Self-host LLMs with vLLM and BentoML
73 stars 12 forks source link

chore: lower gpu memory utilization #62

Closed larme closed 1 month ago