The train_lmdb is 12G,and I use one TITAN Xp to train. I set the batch is 16, if i add the batch size , cuda will report memory error.
Now the GPU utilization is lower than 10%,and the speed is too low,about 40 minutes per 100 iterations.
THANKS!
@wsai2018 It depends on the resolution of the original images. The data loader and online data augmentation are too heavy if the resolution of the original images is too large.
The train_lmdb is 12G,and I use one TITAN Xp to train. I set the batch is 16, if i add the batch size , cuda will report memory error. Now the GPU utilization is lower than 10%,and the speed is too low,about 40 minutes per 100 iterations. THANKS!