RuntimeError: svd_cuda: the updating process of SBDSDC did not converge

facebookresearch / kill-the-bits

Code for: "And the bit goes down: Revisiting the quantization of neural networks"

Other

636 stars 124 forks source link

RuntimeError: svd_cuda: the updating process of SBDSDC did not converge #13

Closed alanMachineLeraning closed 5 years ago

alanMachineLeraning commented 5 years ago

File "/home/zhang/kill-the-bits/src/quantization/em.py", line 83, in step A_pinv = torch.pinverse(in_activations) RuntimeError: svd_cuda: the updating process of SBDSDC did not converge (error: 8) environment：cuda9.0 nvidia-smi:410.93 ubuntu16.04

pierrestock commented 5 years ago

Hello alanMachineLeraning,

It seems that PyTorch had troubles computing the SVD (which is used as a cheap workaround to calculate the pseudo-inverse). Could you provide in_activations.size()? Also, could you try changing --n-activations and/or the number of samples to draw in the function dynamic_sampling (cf. utils/)?

alanMachineLeraning commented 5 years ago

Thank you! Solved! @pierrestock

sayef commented 4 years ago

Thank you! Solved! @pierrestock

@alanMachineLeraning How did you solve the issue? I am also having the same issue.