HolyBayes / pytorch_ard

Pytorch implementation of Variational Dropout Sparsifies Deep Neural Networks
MIT License
83 stars 16 forks source link

Question about speeding up inference time. #9

Open atonkamanda opened 1 year ago

atonkamanda commented 1 year ago

I am confused about inference time reduction, since when I run both the MNIST baseline and the MNIST ARD, the inference time is similar for the first 30 epochs.

I am confused about this sentence in the readme. "Model's sparsification takes almost no any speed-up effects until You convert it to the sparse one! (TODO)"

In my run the model is 99% compressed so pretty sparse and still achieve same inference speed up as the baseline.

Also, I am not sure about what "TODO" mean in that context, does it mean that it is not implemented yet in the repo ?

Thank you.

atonkamanda commented 1 year ago

After reading the code I think a more explicit version of my question is why should we expect some inference computation time improvement when the high dropout rates weights are masked but not dropped and still contained in the computation.

It seems like to gain this computational efficiency the code should be using something like this https://pytorch.org/docs/stable/sparse.html or am I missing something ?