kvcache-ai / ktransformers

A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations
Apache License 2.0
741 stars 39 forks source link

[fix] Fix bugs about static cache and server param; #48

Closed Azure-Tang closed 3 months ago

Azure-Tang commented 3 months ago

fix server don't accept yaml path as param; fix server static cache device problem;