search

hustzxd / LSQuantization

The PyTorch implementation of Learned Step size Quantization (LSQ) in ICLR2020 (unofficial)
123 stars 21 forks source link

Heavy cpu usage for the backward pass in method 1 (FunLSQ) #3

Open haolibai opened 3 years ago

haolibai commented 3 years ago

In my case, the following code seems to consume heavy cpu usage during the backward pass in FunLSQ. (>1000% cpu usage). 

indicate_middle = torch.ones(indicate_small.shape).to(indicate_small.device) - indicate_small - indicate_big

When I turn it to

indicate_middle = 1.0 - indicate_small - indicate_big

, the problem gets solved.

I wonder whether this is also a case for other models in your case. If yes, this could be a reason to explain why method 2 is slower than method 1.

hustzxd commented 3 years ago

Thanks for your advice. I will update the code afterwise.