search

ReaLLMASIC / nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.
MIT License
24 stars 18 forks source link

FiReLU output zeros statistics #269

Open mmoffatt2 opened 2 months ago

mmoffatt2 commented 2 months ago

prints out ReLU output zeros statistics for each layer at every io interval