Anyone tries to train with gpt-j?

young-geng / EasyLM

Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.

Apache License 2.0

2.33k stars 247 forks source link

Closed Taekyoon closed 1 year ago

Taekyoon commented 1 year ago

I'm planning to train gpt-j using tpu v2 resource. Hoping that I can initiate PR for training gpt-j.

young-geng commented 1 year ago

I've trained GPT-J a while ago and it should work. Are you encountering any issues?

Taekyoon commented 1 year ago

No, I could not find guidelines for training gpt-j, so I just made this issue. I'll make another issue if some issues come up :)

Thanks!