search

yanghr / BSQ

BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantization (ICLR 2021)
https://openreview.net/forum?id=TiXl51SCNw8
Apache License 2.0
36 stars 9 forks source link

About CatBackward Error of loss.backward() after net.pruning() #3

Open hadhe145 opened 1 year ago

hadhe145 commented 1 year ago

Thanks for your and your team's contribution!

Here is a bug I met when I used your codes in my own work. As shown in the detail below, I met the incompatible shape error after the network pruning.

I have checked the related codes, when the "nonz_weight[i]==0", this operator will cause the dimension to decrease.

I have added the "del optimizer" and created a new optimizer for the new net,parameters(), but it does not work as I supposed. I am very much looking forward to any relevant help and replies about this problem!

Detailed information about this error loss.backward() File "/home/xxx/lib/python3.8/site-packages/torch/tensor.py", line 198, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph) File "/home/xxx/lib/python3.8/site-packages/torch/autograd/init.py", line 98, in backward Variable._execution_engine.run_backward( RuntimeError: Function CatBackward returned an invalid gradient at index 0 - got [84, 256, 3, 3, 6] but expected shape compatible with [84, 256, 3, 3, 8] (validate_outputs at /pytorch/torch/csrc/autograd/engine.cpp:472) frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x46 (0x7f1b48c7a536 in /home/xxx/lib/python3.8/site-packages/torch/lib/libc10.so) frame #1: + 0x2d84b24 (0x7f1a9f2b9b24 in /home/xxx/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so) frame #2: torch::autograd::Engine::evaluate_function(std::shared_ptr&, torch::autograd::Node*, torch::autograd::InputBuffer&) + 0x548 (0x7f1a9f2bad58 in /home/hp/torch1.5_py38_xyf/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so) frame #3: torch::autograd::Engine::thread_main(std::shared_ptr const&, bool) + 0x3d2 (0x7f1a9f2bcce2 in /home/xxx/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so) frame #4: torch::autograd::Engine::thread_init(int) + 0x39 (0x7f1a9f2b5359 in /home/xxx/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so) frame #5: torch::autograd::python::PythonEngine::thread_init(int) + 0x38 (0x7f1b497f5548 in /home/xxx/lib/python3.8/site-packages/torch/lib/libtorch_python.so) frame #6: + 0xd6d84 (0x7f1b4a6a4d84 in /lib/x86_64-linux-gnu/libstdc++.so.6) frame #7: + 0x9609 (0x7f1b4eff1609 in /lib/x86_64-linux-gnu/libpthread.so.0) frame #8: clone + 0x43 (0x7f1b4f12d293 in /lib/x86_64-linux-gnu/libc.so.6)