Not using GPU during validation

victoresque / pytorch-template

PyTorch deep learning projects made easy.

MIT License

4.75k stars 1.09k forks source link

Not using GPU during validation #60

Closed aleXiehta closed 5 years ago

aleXiehta commented 5 years ago

As title, please check.

SunQpark commented 5 years ago

Are you using nvidia-smi for checking GPU usage?? For small models like MNIST example in this template, GPU-utilization in nvidia-smi is likely to display 0% although GPU is being properly used. On my simple test(printing 'data.device') data and model appeared to be properly on the gpu.

SunQpark commented 5 years ago

Please give me more details, than I'll look more into this issue.

aleXiehta commented 5 years ago

It seems like Tensorboard is slowing down the validation process. GPU usage is normal after removal of the summary writer in for loop. Thanks for the reply.