vectorch-ai / ScaleLLM

A high-performance inference system for large language models, designed for production environments.
https://docs.vectorch.com/
Apache License 2.0
316 stars 23 forks source link

bugfix: fix invalid max_cache_size when device is cpu. #259

Closed liutongxuan closed 3 days ago