search

certik / fastGPT

Fast GPT-2 inference written in Fortran
MIT License
180 stars 16 forks source link

Do an explicit copy of the kv_cache #58

Closed certik closed 1 year ago

certik commented 1 year ago

Compilers have to do a copy of the kv_cache, so let's do it explicitly, which I think is better.

TODO: extract the latest commit, and create a clean commit to upstream.

certik commented 1 year ago

Implemented in https://github.com/certik/fastGPT/pull/60.