Open qwenzo opened 3 months ago
Good point, and it should be. I use GPT-2 myself privately a lot as well, and it'd be nice to have it in LitGPT as well.
I think the architecture is similar to GPTNeo, so you can probably copy and adapt the GPTNeo config. The general todo list I use for adding new configs is:
generate.py
produces reasonable outputs
Hello,
I was wondering if it is straightforward to bring older models such as GPT-2 to lit-gpt. If so, what files/configs do I need to change?
Thank you!