Add option to use SwiGLU FFN Option - Githubissues

ReaLLMASIC / nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.

MIT License

23 stars 17 forks source link

Add option to use SwiGLU FFN Option #168

Closed gkielian closed 3 months ago