Open YannPourcenoux opened 2 years ago
@rino20 Hi Rino, could you help fix this issue?
Can I get an update on this? This is one of your tutorials and 1x1 Convs are one of the most used layers in Deep Learning for computer vision
@wwwind Could you take a look? Thanks.
Hi @rino20 Yes, we will take a look today/tomorrow at this issue.
Hi @YannPourcenoux I'm taking a look at this now, also. Will keep you posted.
@YannPourcenoux We have found the source of the problem and are now working towards a solution.
Great! Thanks! Looking forward to hearing from you again 😁
Hi @YannPourcenoux a PR has been created for the issue (linked above). In the meantime, if you like, while waiting for it to be merged, you may download the patch and see for yourself if you get the desired behaviour with the 1x1 kernel sizes. Thanks for drawing attention to this bug :)
Describe the bug When doing the Sparsity and cluster preserving quantization aware training (PCQAT) Keras example, if I use a Conv2D Layer with a kernel size of
(1, 1)
the model after the QAT step of PCQAT has only zeros in this weight. It works fine if the kernel size is(3, 3)
or bigger.System information
TensorFlow version (installed from source or binary): 2.9.1 from
pip
TensorFlow Model Optimization version (installed from source or binary): 0.7.2 from
pip
Python version: 3.9.12
Describe the expected behavior Having the same behavior whether the kernel size of the convolutional layer is
(1, 1)
or(3, 3)
.Describe the current behavior The final model only has 0 in the weights after doing the QAT step which preserves sparsity and clustering. The sparsity is 100% as shown by the
print()
Code to reproduce the issue