ACIQ clipping not in sync with paper

[amitsrivastava78]

Hi @ynahshan , @submission2019, I checked your paper and code, so as per my understanding the after forward pass from each operator (example conv), there will be ACIQ based on the laplase (the input i had used for 4-bit quantizing the resnet50). So i assumed that the clipping will only clip the regions from left and right of the tensors, but when i plot the histogram i can see entirly different histogram for output, I am attaching one the histogram as reference to you, the Red line shows the input tensor and black one shows the output(clipped tensor). Can you please explain the result which is not in sync with your paper or at least my understanding of your paper.

Regards Amit

[amitsrivastava78]

Also there is another histogram from the same flow which show slight traces but still not same as per the paper. Attaching the same below as mentioned above red is the input tensor after the opeator forward pass and black line represents the ACIQ tensor.

Can you please let me know why i am i seeing this kind of behaviour

Regards Amit

[submission2019]

Hi @amitsrivastava78. ACIQ clipping applying as part of quantization procedure. In order to observe it's effect on tensor level you should dump data before and after quantization. In case of convolution it could be done in inference_quantization_manager.py line 208 like this. ` torch.save(out, os.path.join('dump', activation_id + '.pt'))

out = QMI().quantize_instant(out, tag_act, half_range=hasattr(self, 'before_relu'), verbose=QMI().verbose)

torch.save(out, os.path.join('dump', activation_id + '_q.pt')) `

Also note that quantization and ACIQ applied on every channel separately. Here is some histograms that I did on arbitrary layers and channels. IMHO it is hard to see the effect of ACIQ on histogram and it is better look into statistics of all layers of min/max with and w/o ACIQ.

[amitsrivastava78]

@submission2019 , @ynahshan , thanks for the response, i had put the code exactly where you asked me to put but still i did not observe proper ACIQ, rather the histogram came exactly what i had posted in my first two queries, please find the code snippet and histogram as below : -

Kindly help me to understand why is this happening ? Am i missing something here ?

Regards Amit

[submission2019]

Hi. There is few problems with you code.

1. torch.histc() returns only counts of elements per bin with out any adges information so X axis of your plot always 50 with regards to the range of tensor.
2. Even with plt.hist() it is hard to see the range due to large amount of elements.

If you really want to measure range due to clipping the best way is just to print it like this: print(out.max() - out.min())

But make sure you handle relu in cases where quantization is following relu.

if hasattr(self, 'before_relu'):
    out = torch.nn.functional.relu(out)

[submission2019]

Hi @amitsrivastava78. Do you have any more questions related to this topic? If not I will close this issue.