ReaLLMASIC / nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.
MIT License
23 stars 17 forks source link

Add MLP Expansion factor control and sweep #252

Closed gkielian closed 2 months ago

gkielian commented 2 months ago

Some networks have started experimenting with different expansion factors, here we add a sweep for this testing affects of different mlp settings.