Open sidgairo18 opened 5 years ago
To accelerate training step, you can fix the parameters in the backbone model or input the features instead of the images. If you have two or more GPUs, you can finish training the model within one day.
Thank you for your reply. I am currently using a Batch-Size of 20 with 4 GPUs and training the model end to end.
I have 500K pairs for training. It takes 1 day for a single epoch with batch size 20.
The training is very slow. Even the evaluate part takes hours to finish.
How much time did it take to train the model?