search

intel / xFasterTransformer

Apache License 2.0
344 stars 60 forks source link

[KVCache] KVCache and KVCacheMgr refactor to support continuous batching. #329

Closed pujiang2018 closed 4 months ago

pujiang2018 commented 4 months ago

close as will put to cb_dev branch.