how to frozen some layers of GPT, only fintune last k layers?

bigscience-workshop / Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Other

1.3k stars 211 forks source link

Open joan126 opened 1 year ago

joan126 commented 1 year ago

Hello, I want to finetune only last k layer of GPT model. how can I solve this?