haileyschoelkopf
commented
7 months ago Hi! Please see the Pythia, page 18 for notes on this--we renamed the model sizes to count the embed+unembed parameters in the models.
Closed borgr closed 7 months ago
haileyschoelkopf
commented
7 months ago Hi! Please see the Pythia, page 18 for notes on this--we renamed the model sizes to count the embed+unembed parameters in the models.
In the evals/bias-evals/ dir there are files starting with pythia 350m and pythia 1.3B, but those aren't sizes of pythia v0 nor v1, right? So is it 410m and 1.4B or is it not pythia? Or am I missing something in the experimental setting?