[KVCache] Unlimited depth blocks

apache / tvm

Open deep learning compiler stack for cpu, gpu and specialized accelerators

https://tvm.apache.org/

Apache License 2.0

11.41k stars 3.4k forks source link

Closed cyx-6 closed 2 weeks ago

cyx-6 commented 2 weeks ago

This PR updates the kvcache to support the unlimited depth blocks.

cc: @tqchen @MasterJH5574