When I evaluate the quantized model, I got all 0s inference result for test samples.
When I run evaluation, it goes into
if dev.simulate:
condition, but would you need below function when the model is already quantized?
self.quantize = Quantize(num_bits=dev.DATA_BITS)
This function tries to set the value to either 0 or 1. Then the values are inherited to the latter layers, even though they expect values to be between -128 to +127.
When I set
self.quantize = Empty()
It seems it's working correctly, but I'm afraid if it's the right solution.
It seems it's already discussed in #305, however,
When I evaluate the quantized model, I got all 0s inference result for test samples. When I run evaluation, it goes into
if dev.simulate:
condition, but would you need below function when the model is already quantized?
self.quantize = Quantize(num_bits=dev.DATA_BITS)
This function tries to set the value to either 0 or 1. Then the values are inherited to the latter layers, even though they expect values to be between -128 to +127.
When I set
self.quantize = Empty()
It seems it's working correctly, but I'm afraid if it's the right solution.