search

Xirider / finetune-gpt2xl

Guide: Finetune GPT2-XL (1.5 Billion Parameters) and finetune GPT-NEO (2.7 B) on a single GPU with Huggingface Transformers using DeepSpeed
MIT License
431 stars 73 forks source link

fine tuning GPT-J 6B? #22

Open silvacarl2 opened 1 year ago

silvacarl2 commented 1 year ago

hi, this is not an issue but i was not sure where to post it.

how can this tool be adapted for fine tuning GPT-J 6B?

jaimu97 commented 1 year ago

I am using this repo which took inspiration from here

https://github.com/mallorbc/Finetune_GPTNEO_GPTJ6B/tree/main/finetuning_repo