Add flexible padding bonus experiment

rasbt / LLMs-from-scratch

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

Other

34.13k stars 4.18k forks source link

Closed rasbt closed 2 weeks ago

rasbt commented 2 weeks ago

Adds an additional experiment with a flexible batch size to avoid using padding tokens. For more details, please see discussion #434.