Pretrained model hyperparameters

[ladanisavan]

Hi @ku21fan Just want to know which hyperparameters (no. of iterations, lr, batch size, optimizer) are used to train TPS-ResNet-BiLSTM-Attn-case-sensitive model? also what was the final training loss and validation loss? did you decrease lr during your training? thanks

[ku21fan]

Hello,

For TPS-ResNet-BiLSTM-Attn-case-sensitive model, I used the same setting in this repo.

• no. of iterations: 300K
• lr: 1
• batch size: 192
• optimizer: adadelta

I don't have log file now... so it is hard to answer the final training/validation loss I did not decrease lr during training.

Hope it helps, Best.

[ladanisavan]

Thanks for your quick reply!!

I'm training with adam optimizer and here is current statistics of my model training. Do you recommend to decrease learning rate?

• itr_num: 150000/300000
• train loss: 0.05675
• valid loss: 2.35073
• lr: 0.001
• best_accuracy: 64.974

[ku21fan]

Yes, decreasing the learning rate would be helpful :)

[lvforce]

@ku21fan I have a pre-trained model from EasyOCR(None-VGG-BiLSTM-CTC), and I want to retrain it on my own data.

Question: Should I freeze FeatureExtraction and SequenceModeling part and just fine-tune CTC and use -FT flag?