kvcache-ai / ktransformers

A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations
Apache License 2.0
729 stars 37 forks source link

[fix] Fix bugs about static cache and server param; #48

Closed Azure-Tang closed 2 months ago

Azure-Tang commented 2 months ago

fix server don't accept yaml path as param; fix server static cache device problem;