Add KV cache option for inference

alan-turing-institute / minGPT

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

MIT License

1 stars 0 forks source link

Add KV cache option for inference #10

Closed rchan26 closed 3 weeks ago

rchan26 commented 1 month ago

Probably need to review #9 first (just refactoring stuff)

Fix #1

review-notebook-app[bot] commented 1 month ago

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

crangelsmith commented 1 month ago

The code looks good to me and I've run it locally on my machine. @rchan26 suggested he take us through the implementation in next week's Karpathy session.