Open atonkamanda opened 1 year ago
After reading the code I think a more explicit version of my question is why should we expect some inference computation time improvement when the high dropout rates weights are masked but not dropped and still contained in the computation.
It seems like to gain this computational efficiency the code should be using something like this https://pytorch.org/docs/stable/sparse.html or am I missing something ?
I am confused about inference time reduction, since when I run both the MNIST baseline and the MNIST ARD, the inference time is similar for the first 30 epochs.
I am confused about this sentence in the readme. "Model's sparsification takes almost no any speed-up effects until You convert it to the sparse one! (TODO)"
In my run the model is 99% compressed so pretty sparse and still achieve same inference speed up as the baseline.
Also, I am not sure about what "TODO" mean in that context, does it mean that it is not implemented yet in the repo ?
Thank you.