search

EleutherAI / pythia

The hub for EleutherAI's work on interpretability and learning dynamics
Apache License 2.0
2.16k stars 156 forks source link

Optimizer states in HF format #152

Open seyuboglu opened 4 months ago

seyuboglu commented 4 months ago

Hi folks -- thanks for the great work on this.

I've been doing some fine-tuning experiments off the huggingface checkpoints and was wondering whether anyone has converted the neox optimizer states to HF format/has a script for doing so.

Thanks!!

haileyschoelkopf commented 4 months ago

I don't know of any such preexisting conversion script unfortunately -- but if you have questions in creating those or about the checkpoint format lmk!