Closed certik closed 1 year ago
Compilers have to do a copy of the kv_cache, so let's do it explicitly, which I think is better.
TODO: extract the latest commit, and create a clean commit to upstream.
Implemented in https://github.com/certik/fastGPT/pull/60.
Compilers have to do a copy of the kv_cache, so let's do it explicitly, which I think is better.
TODO: extract the latest commit, and create a clean commit to upstream.