search

IntelLabs / distiller

Neural Network Distiller by Intel AI Lab: a Python package for neural network compression research. https://intellabs.github.io/distiller
Apache License 2.0
4.34k stars 800 forks source link

ValueError when using QAT-PACT #524

Open jlimmm opened 4 years ago

jlimmm commented 4 years ago

ValueError occurs when I run quantization-aware-training using PACT. The error message is "ValueError: saturation_min must be smaller than saturation_max" and it seems like the error occurs when the clipping value of ReLU is below zero because the clipping value cannot be negative theoretically.

setting:

================= Error Log ============================ File "compress_classifier.py", line 212, in main() File "compress_classifier.py", line 78, in main app.run_training_loop() File "/home/jelim/f_github/test_distiller/distiller/distiller/apputils/image_classifier.py", line 192, in run_training_loop top1, top5, loss = self.train_validate_with_scheduling(epoch) File "/home/jelim/f_github/test_distiller/distiller/distiller/apputils/image_classifier.py", line 130, in train_validate_with_scheduling top1, top5, loss = self.train_one_epoch(epoch, verbose) File "/home/jelim/f_github/test_distiller/distiller/distiller/apputils/image_classifier.py", line 116, in train_one_epoch loggers=[self.tflogger, self.pylogger], args=self.args) File "/home/jelim/f_github/test_distiller/distiller/distiller/apputils/image_classifier.py", line 577, in train output = model(inputs) File "/Data2/home/jelim/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(*input, kwargs) File "/Data2/home/jelim/.local/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward return self.module(*inputs[0], *kwargs[0]) File "/Data2/home/jelim/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(input, kwargs) File "/home/jelim/f_github/test_distiller/distiller/distiller/models/cifar10/preresnet_cifar.py", line 153, in forward x = self.layer1(x) File "/Data2/home/jelim/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(*input, kwargs) File "/Data2/home/jelim/.local/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward input = module(input) File "/Data2/home/jelim/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(*input, *kwargs) File "/home/jelim/f_github/test_distiller/distiller/distiller/models/cifar10/preresnet_cifar.py", line 76, in forward out = self.relu(out) File "/Data2/home/jelim/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(input, kwargs) File "/home/jelim/f_github/test_distiller/distiller/distiller/quantization/clipped_linear.py", line 65, in forward scale, zero_point = asymmetric_linear_quantization_params(self.num_bits, 0, self.clip_val, signed=False) File "/home/jelim/f_github/test_distiller/distiller/distiller/quantization/q_utils.py", line 91, in asymmetric_linear_quantization_params raise ValueError('saturation_min must be smaller than saturation_max') ValueError: saturation_min must be smaller than saturation_max

====================================================

error_saturation_min_PACT.log pact_yaml.zip

CosimoRulli commented 3 years ago

Hi, did you find a solution to this issue?