[Question] Why is GPU utilization jumping so hard with NASNetMobile?

keras-team / keras-applications

Reference implementations of popular deep learning models.

Other

2k stars 913 forks source link

[Question] Why is GPU utilization jumping so hard with NASNetMobile? #146

Open PhilipMay opened 4 years ago

PhilipMay commented 4 years ago

I am using a NASNetMobile. nvidia-smi somehow shows that GPU utilization is jumping between 12% and 95%. With mean somewhere between. I know from other trainings that GPU utilization is not 100% stable. But these high jumps are extraordinary.

I am using a 1080 Ti with batch size of 64. The images are 150 by 150 pixel.

Can anybody explain this?