what is the learning rate should be when finetune a small dataset on a base model

descriptinc / melgan-neurips

GAN-based Mel-Spectrogram Inversion Network for Text-to-Speech Synthesis

MIT License

978 stars 214 forks source link

what is the learning rate should be when finetune a small dataset on a base model #37

Open Liujingxiu23 opened 3 years ago

Liujingxiu23 commented 3 years ago

I have trained a base model using 4 female datasets using the followding lr. optD = torch.optim.Adam(netD.parameters(), lr=1e-4, betas=(0.5, 0.9))

Then I want to finetune the model using a new small female dataset, should I change the lr. Maybe lr=0.00001 without decay? Does anyone have done the experiements and have any conclusion?