openspyrit / spyrit

A Python toolbox for deep image reconstruction, with emphasis on single-pixel imaging.
https://spyrit.readthedocs.io/en/master
GNU Lesser General Public License v3.0
38 stars 15 forks source link

Training : GPU usage fluctuating #222

Open romainphan opened 1 month ago

romainphan commented 1 month ago

Is there a way to always use as much GPU as possible during training phase ? Same with different cores of the CPU ?

romainphan commented 1 month ago

Maybe : https://discuss.pytorch.org/t/unstable-gpu-utilization-when-training-resnet/143507/7 --> https://on-demand.gputechconf.com/gtc-cn/2019/pdf/CN9951/presentation.pdf --> https://github.com/NVIDIA/nsight-training

nducros commented 1 month ago

Usually I tend to increase the batch size till I reach the maximum memory available. Not sure this answers your question...