training

[nlm676]

Hello, did you set the batch_size to 3 according to the configuration file and train for 1000 epochs, I found that I need to train for about 20 days, and I see that your weights have only been trained for 14 hours, looking forward to your reply

[ZhangAoCanada]

That's odd. If my memory is right, the model overfit after training more than 3 days during my experiments, and my training device was a single 2080ti.

Do you mind to tell me what your training configurations are and what kind of GPUs you are working on?

[nlm676]

My configuration is unchanged from yours, batch_size is set to 3, epoch is set to 1000, and I train with a single 1080ti

[ZhangAoCanada]

That looks normal, cannot figure out why.....

[wenyiwangbst]

That looks normal, cannot figure out why.....

Hi, Can I ask how many epochs did you train before it overfit? In my experiment it overfit by only 8 epochs

[ShamsAuda]

Hi I also run the same code on the same data after setting the epochs to 20 and it took 126 hours which is way far from your time. Could you please give access to the pretrained model checkpoints as the link provided does not work.

[IqbalBan]

Has anybody been able to reproduce the results while training for less than 1000 epochs? The training time for me is quite slow, and training for all 1000 epochs would take an unreasonable amount of time.