search

Swall0w / torchstat

Model analyzer in PyTorch
MIT License
1.47k stars 142 forks source link

How does torchstat compute total memory? #19

Open un-knight opened 5 years ago

un-knight commented 5 years ago

How does torchstat compute total memory for a model?

For example, I use torchstat to analyze mobilenet and get the following results:

Total params: 4,231,976
--------------------------------------
Total memory: 38.48MB

Since the total params are 4,231,976, so I think the total params size is 4,231,976 * 4 / (1024^2) = 16.14 MB, while torchstat gives a total memory 38.48 MB which makes me really confused.

DocF commented 5 years ago

hi，guy. I think the total memory term is the sum of all output feature maps of every operation (e.g. convs, relu, BN ... ). In some ways, the sum equal to the memory-write number.

hychiang-git commented 2 years ago

Shall we also consider the parameters loaded into memory and do mul-add with feature maps? The number should be equal to MemRead.