search

kvcache-ai / ktransformers

A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations
Apache License 2.0
741 stars 39 forks source link

how to implement new algorithm in this repo? #105

Closed lumiere-ml closed 3 weeks ago

lumiere-ml commented 3 weeks ago

now confused with this repo architech, want to know how can I implement new algo in this repo, such as pyramidKV

Azure-Tang commented 3 weeks ago

Hi, maybe this doc can help you https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/injection_tutorial.md