Introduce buffers to improve Llama 3.2 efficiency

rasbt / LLMs-from-scratch

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

Other

34.28k stars 4.2k forks source link

Closed rasbt closed 1 month ago

rasbt commented 1 month ago

Adds shared buffers to avoid recreating the mask and cos, sin values in each transformer block

review-notebook-app[bot] commented 1 month ago

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.