search

ReaLLMASIC / nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.
MIT License
24 stars 18 forks source link

Refactor to add activation config #294

Closed gkielian closed 3 weeks ago

gkielian commented 3 weeks ago

Currently this has the activations as different per layer, need to add and modify the sharing settings.

(Also should add more variations for the sharing options, including roundrobin sharing)