Open sikaleo opened 4 years ago
hi, i'm trying to load pretrained model's checkpoint and pretrain other corpus. But, it's mlm acc decreased a lot.
1) So, Is it impossible to load pretrained model's checkpoint and pretrain other corpus with that checkpoint?
2) If we load pretrained model's checkpoint and use it, is only fine tuning possible?
hi, i'm trying to load pretrained model's checkpoint and pretrain other corpus. But, it's mlm acc decreased a lot.
1) So, Is it impossible to load pretrained model's checkpoint and pretrain other corpus with that checkpoint?
2) If we load pretrained model's checkpoint and use it, is only fine tuning possible?