search

hoya012 / automatic-mixed-precision-tutorials-pytorch

Automatic Mixed Precision Tutorials using pytorch. Based on PyTorch 1.6 Official Features, implement classification codebase using custom dataset.
MIT License
87 stars 6 forks source link

Memory usage was not decreased when using amp #1

Open joshhu opened 3 years ago

joshhu commented 3 years ago

Hi,

I was trying your code couple of days ago. It turned out when i used the '--amp' parameter, the memory usage was not decreased (around 10G of of my 2080Ti 11G). The training time was also almost the same without amp.

How did you get the results?

Thank you.

hoya012 commented 3 years ago

Hi, Thanks for opening the Issue. I have had a similar experience with Video Recognition. Memory usage may decrease when an operation optimized for the AMP function is used, but otherwise, the effect may not be seen.

For detailed cause analysis, it seems to be necessary to check the code and architecture used in the experiment. Thanks!