train mix bit[4,8],but get the model look like int8

betterhalfwzm commented 3 years ago

trained model:

ljaljushkin commented 3 years ago

Greetings, @betterhalfwzm!

As far as I can see from your config, there's no target_device and by default it's CPU. Currently, CPU supports only int8 quantizations, that's why mixed-precision initialization wasn't applied.

For mixed precision configuration you should specify VPU or TRIAL device:

"target_device": "VPU",
"compression": {
  ...
}

betterhalfwzm commented 3 years ago

Greetings, @betterhalfwzm!

As far as I can see from your config, there's no target_device and by default it's CPU. Currently, CPU supports only int8 quantizations, that's why mixed-precision initialization wasn't applied.

For mixed precision configuration you should specify VPU or TRIAL device:
"target_device": "VPU",
"compression": {
  ...
}

@ljaljushkin Thank you for your reply. When i use this config,training report error, use "type": "manual", training is normal. Config: Error: INFO:nncf:Initialization of quantization precisions Traceback (most recent call last): File "main_MobileNetV2_145M_threechannels_nncf_fold_BN.py", line 119, in compression_ctrl, compressed_model = create_compressed_model(model, nncf_config) File "/home/wangzhaoming/anaconda3/envs/torch181/lib/python3.7/site-packages/nncf/model_creation.py", line 122, in create_compressed_model composite_builder.apply_to(compressed_model) File "/home/wangzhaoming/anaconda3/envs/torch181/lib/python3.7/site-packages/nncf/composite_compression.py", line 78, in apply_to layout = self.get_transformation_layout(target_model) File "/home/wangzhaoming/anaconda3/envs/torch181/lib/python3.7/site-packages/nncf/composite_compression.py", line 111, in get_transformation_layout transformations.update(builder.get_transformation_layout(model)) File "/home/wangzhaoming/anaconda3/envs/torch181/lib/python3.7/site-packages/nncf/compression_method_api.py", line 225, in get_transformation_layout layout = self._get_transformation_layout(target_model) File "/home/wangzhaoming/anaconda3/envs/torch181/lib/python3.7/site-packages/nncf/quantization/algo.py", line 607, in _get_transformation_layout single_config_quantizer_setup = self._get_quantizer_setup(target_model) File "/home/wangzhaoming/anaconda3/envs/torch181/lib/python3.7/site-packages/nncf/quantization/algo.py", line 686, in _get_quantizer_setup single_config_quantizer_setup = setup_generator.generate_setup() File "/home/wangzhaoming/anaconda3/envs/torch181/lib/python3.7/site-packages/nncf/quantization/algo.py", line 363, in generate_setup quantization_proposal.quantizer_setup) File "/home/wangzhaoming/anaconda3/envs/torch181/lib/python3.7/site-packages/nncf/quantization/algo.py", line 300, in select_final_quantizer_setup hw_constraints) File "/home/wangzhaoming/anaconda3/envs/torch181/lib/python3.7/site-packages/nncf/quantization/algo.py", line 1247, in init_precision return initializer.apply_init() File "/home/wangzhaoming/anaconda3/envs/torch181/lib/python3.7/site-packages/nncf/quantization/precision_init/hawq_init.py", line 263, in apply_init traces_per_layer = self._calc_traces(self._criterion_fn, self._criterion, self._iter_number, self._tolerance) File "/home/wangzhaoming/anaconda3/envs/torch181/lib/python3.7/site-packages/nncf/quantization/precision_init/hawq_init.py", line 460, in _calc_traces avg_traces = trace_estimator.get_average_traces(max_iter=iter_number, tolerance=tolerance) File "/home/wangzhaoming/anaconda3/envs/torch181/lib/python3.7/site-packages/nncf/quantization/hessian_trace.py", line 127, in get_average_traces avg_traces_per_iter.append(self._calc_avg_traces_per_param()) File "/home/wangzhaoming/anaconda3/envs/torch181/lib/python3.7/site-packages/nncf/quantization/hessian_trace.py", line 144, in _calc_avg_traces_per_param for gradients in self._gradients_calculator: File "/home/wangzhaoming/anaconda3/envs/torch181/lib/python3.7/site-packages/nncf/quantization/hessian_trace.py", line 89, in next loss.backward(create_graph=True) AttributeError: 'tuple' object has no attribute 'backward'

ljaljushkin commented 3 years ago

HAWQ algorithm calculates importance of quantization precisions by running back propagation multiple times, so it requires registration of valid criterion function to do it. By default criterion is Cross Entropy: https://github.com/openvinotoolkit/nncf/blob/5f8c1329be6f40f8fbdc6aad7538aaf6d5b6c239/examples/torch/classification/main.py#L126

This criterion and function that calculates loss for a given output and target are registered by register_default_init_args call: https://github.com/openvinotoolkit/nncf/blob/5f8c1329be6f40f8fbdc6aad7538aaf6d5b6c239/examples/torch/classification/main.py#L148-L150

The error happens because default loss function https://github.com/openvinotoolkit/nncf/blob/5f8c1329be6f40f8fbdc6aad7538aaf6d5b6c239/nncf/torch/initialization.py#L247-L248 is not suitable for your model. We expect that loss function returns a single Tensor, not a Tuple. You can register custom criterion function that produces loss in the expected format.

betterhalfwzm commented 3 years ago

HAWQ algorithm calculates importance of quantization precisions by running back propagation multiple times, so it requires registration of valid criterion function to do it. By default criterion is Cross Entropy:

https://github.com/openvinotoolkit/nncf/blob/5f8c1329be6f40f8fbdc6aad7538aaf6d5b6c239/examples/torch/classification/main.py#L126

This criterion and function that calculates loss for a given output and target are registered by register_default_init_args call:

https://github.com/openvinotoolkit/nncf/blob/5f8c1329be6f40f8fbdc6aad7538aaf6d5b6c239/examples/torch/classification/main.py#L148-L150

The error happens because default loss function

https://github.com/openvinotoolkit/nncf/blob/5f8c1329be6f40f8fbdc6aad7538aaf6d5b6c239/nncf/torch/initialization.py#L247-L248

is not suitable for your model. We expect that loss function returns a single Tensor, not a Tuple. You can register custom criterion function that produces loss in the expected format.

@ljaljushkin Thank you for your reply. i add criterion_fn，train log is this，is right？ INFO:nncf:Initialization of quantization precisions INFO:nncf:0# difference_avg=nan avg_trace=nan INFO:nncf:1# difference_avg=nan avg_trace=nan INFO:nncf:2# difference_avg=nan avg_trace=nan INFO:nncf:3# difference_avg=nan avg_trace=nan INFO:nncf:4# difference_avg=nan avg_trace=nan INFO:nncf:Chosen HAWQ bitwidth sequence with ratio=1.53, bitwidth per weightable layer=[4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8]

ljaljushkin commented 3 years ago

It doesn't seem right. Please make sure that compression is applied for a pre-trained model, not for a model from scratch. It's necessary for a correct work of precision initialization. For the models from torchvision you can specify "pretrained" property in the config (like in the example). Loading of custom pre-trained weights can be done via --weights argument for the sample.

If it doesn't solve the issue, you could enable collection of debug information and send content of the nncf_debug/hawq_dumps folder as an archive. For that add the "dump_init_precision_data" parameter to initialization section:

...
 "compression": {
        "algorithm": "quantization",
       ...
        "initializer": {
            "precision": {
                "dump_init_precision_data": True,
                ...
            }
        }
}

betterhalfwzm commented 3 years ago

It doesn't seem right. Please make sure that compression is applied for a pre-trained model, not for a model from scratch. It's necessary for a correct work of precision initialization. For the models from torchvision you can specify "pretrained" property in the config (like in the example). Loading of custom pre-trained weights can be done via --weights argument for the sample.

If it doesn't solve the issue, you could enable collection of debug information and send content of the nncf_debug/hawq_dumps folder as an archive. For that add the "dump_init_precision_data" parameter to initialization section:
...
 "compression": {
        "algorithm": "quantization",
       ...
        "initializer": {
            "precision": {
                "dump_init_precision_data": True,
                ...
            }
        }
} 

@ljaljushkin Is this train log look like normal? Thx!!!!!!!! INFO:nncf:Initialization of quantization precisions INFO:nncf:0# difference_avg=15833921536.0 avg_trace=15833.921875 INFO:nncf:1# difference_avg=0.25250551104545593 avg_trace=19832.07421875 INFO:nncf:2# difference_avg=0.13357524573802948 avg_trace=17183.0 INFO:nncf:3# difference_avg=0.007286225911229849 avg_trace=17057.80078125 INFO:nncf:4# difference_avg=0.0669918954372406 avg_trace=15915.06640625 INFO:nncf:5# difference_avg=0.003985757473856211 avg_trace=15978.5 INFO:nncf:6# difference_avg=0.08464193344116211 avg_trace=17330.951171875 INFO:nncf:7# difference_avg=0.0027045859023928642 avg_trace=17377.82421875 INFO:nncf:8# difference_avg=0.008418483659625053 avg_trace=17231.529296875 INFO:nncf:9# difference_avg=0.03570138290524483 avg_trace=17846.71875 INFO:nncf:10# difference_avg=0.0007328027859330177 avg_trace=17833.640625 INFO:nncf:11# difference_avg=0.024255642667412758 avg_trace=18266.20703125 INFO:nncf:12# difference_avg=0.022960346192121506 avg_trace=17846.80859375 INFO:nncf:13# difference_avg=0.01100599393248558 avg_trace=17650.38671875 INFO:nncf:14# difference_avg=0.010162003338336945 avg_trace=17829.75 INFO:nncf:15# difference_avg=0.040701594203710556 avg_trace=18555.44921875 INFO:nncf:16# difference_avg=0.005376621149480343 avg_trace=18455.68359375 INFO:nncf:17# difference_avg=0.028138773515820503 avg_trace=17936.36328125 INFO:nncf:18# difference_avg=0.020061267539858818 avg_trace=18296.189453125 INFO:nncf:19# difference_avg=0.0036096570547670126 avg_trace=18230.146484375 INFO:nncf:20# difference_avg=0.007434993050992489 avg_trace=18094.60546875 INFO:nncf:21# difference_avg=0.06851047277450562 avg_trace=19334.275390625 INFO:nncf:22# difference_avg=0.0027965030167251825 avg_trace=19280.20703125 INFO:nncf:23# difference_avg=0.0012703280663117766 avg_trace=19255.71484375 INFO:nncf:24# difference_avg=0.01609303057193756 avg_trace=18945.83203125 INFO:nncf:25# difference_avg=0.009211912751197815 avg_trace=19120.359375 INFO:nncf:26# difference_avg=0.010499687865376472 avg_trace=19321.1171875 INFO:nncf:27# difference_avg=0.0033862318377941847 avg_trace=19255.69140625 INFO:nncf:28# difference_avg=0.01907532662153244 avg_trace=18888.3828125 INFO:nncf:29# difference_avg=0.006498599890619516 avg_trace=18765.634765625 INFO:nncf:30# difference_avg=0.011457216925919056 avg_trace=18550.6328125 INFO:nncf:31# difference_avg=0.00015877153782639652 avg_trace=18553.578125 INFO:nncf:32# difference_avg=0.0016748369671404362 avg_trace=18522.50390625 INFO:nncf:33# difference_avg=0.010094667784869671 avg_trace=18335.525390625 'INFO:nncf:34# difference_avg=0.003462156979367137 avg_trace=18272.044921875 INFO:nncf:35# difference_avg=0.006328186485916376 avg_trace=18156.416015625 INFO:nncf:36# difference_avg=0.001389509649015963 avg_trace=18131.1875 INFO:nncf:37# difference_avg=0.009715218096971512 avg_trace=18307.3359375 INFO:nncf:38# difference_avg=0.008454709313809872 avg_trace=18152.552734375 INFO:nncf:39# difference_avg=0.0017923185368999839 avg_trace=18185.087890625 INFO:nncf:40# difference_avg=0.012601327151060104 avg_trace=17955.931640625 INFO:nncf:41# difference_avg=0.009322737343609333 avg_trace=17788.533203125 INFO:nncf:42# difference_avg=0.003188281087204814 avg_trace=17731.818359375 INFO:nncf:43# difference_avg=0.009942622855305672 avg_trace=17555.517578125 INFO:nncf:44# difference_avg=0.007925971411168575 avg_trace=17694.662109375 INFO:nncf:45# difference_avg=0.0009988226229324937 avg_trace=17676.98828125 INFO:nncf:Chosen HAWQ bitwidth sequence with ratio=1.52, bitwidth per weightable layer=[8, 8, 8, 8, 4, 8, 8, 4, 4, 8, 4, 8, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4] INFO:nncf:Distribution of HAWQ metrics: min_value=846.199, max_value=101653.734, median_value=25575.303, median_index=26, total_number=51 INFO:nncf: "bitwidth_per_scope": [ [8, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv0_0]'], [8, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv1_0_ds1]'], [8, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv1_0_dw]'], [8, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_0_ds0]'], [8, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_0_ds1]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_0_dw]'], [8, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_1_ds0]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_1_ds1]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_1_dw]'], [8, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_0_ds0]'], [8, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_0_ds1]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_0_dw]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_1_ds0]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_1_ds1]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_1_dw]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_2_ds0]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_2_ds1]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_2_dw]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_0_ds0]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_0_ds1]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_0_dw]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_1_ds0]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_1_ds1]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_1_dw]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_2_ds0]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_2_ds1]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_2_dw]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_3_ds0]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_3_ds1]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_3_dw]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_0_ds0]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_0_ds1]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_0_dw]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_1_ds0]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_1_ds1]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_1_dw]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_2_ds0]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_2_ds1]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_2_dw]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_ds0_det]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_ds1_det]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_ds2_det]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_ds3_det]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_dw2_det]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_dw_det]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv7_0_ds0_det]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv7_0_ds1_det]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv7_0_dw_det]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv_finalout]'], [4, 'TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[deconv_0_ds0]'], [8, 'TargetType.OPERATOR_POST_HOOK /nncf_model_input_0'], [8, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv1_0_ds1]/conv2d_0'], [8, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv2_0_ds1]/conv2d_0'], [8, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv2_1_ds1]/conv2d_0'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv3_0_ds1]/conv2d_0'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv3_1_ds1]/conv2d_0'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv3_2_ds1]/conv2d_0'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv4_0_ds1]/conv2d_0'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv4_1_ds1]/conv2d_0'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv4_2_ds1]/conv2d_0'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv4_3_ds1]/conv2d_0'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv5_0_ds1]/conv2d_0'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv5_1_ds1]/conv2d_0'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv5_2_ds1]/conv2d_0'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_ds1_det]/conv2d_0'], [8, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_0'], [8, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_1'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_10'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_11'], [8, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_12'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_13'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_14'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_15'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_16'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_17'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_18'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_19'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_2'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_20'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_21'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_22'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_23'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_24'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_25'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_26'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_27'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_28'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_29'], [8, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_3'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_30'], [8, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_31'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_32'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_33'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_34'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_4'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_5'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_6'], [8, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_7'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_8'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_9'], [8, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_0'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_1'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_2'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_3'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_4'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_5'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_6'], [4, 'TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_7'] ] INFO:nncf:Collecting tensor statistics ████████ | 1 / 2 INFO:nncf:Collecting tensor statistics ████████████████ | 2 / 2 INFO:nncf:Set sign: False and scale: [17.1771, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_1 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_1 INFO:nncf:Set sign: False and scale: [17.0129, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_3 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_3 INFO:nncf:Set sign: False and scale: [19.7299, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_5 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_5 INFO:nncf:Set sign: True and scale: [13.4859, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv2_1_ds1]/conv2d_0 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv2_1_ds1]/conv2d_0 INFO:nncf:Set sign: False and scale: [10.9829, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_7 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_7 INFO:nncf:Set sign: False and scale: [31.0935, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_9 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_9 INFO:nncf:Set sign: True and scale: [10.3291, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv3_1_ds1]/conv2d_0 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv3_1_ds1]/conv2d_0 INFO:nncf:Set sign: False and scale: [18.1353, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_11 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_11 INFO:nncf:Set sign: True and scale: [15.4170, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv3_2_ds1]/conv2d_0 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv3_2_ds1]/conv2d_0 INFO:nncf:Set sign: False and scale: [19.9188, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_14 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_14 INFO:nncf:Set sign: False and scale: [20.9518, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_16 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_16 INFO:nncf:Set sign: True and scale: [7.7216, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv4_1_ds1]/conv2d_0 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv4_1_ds1]/conv2d_0 INFO:nncf:Set sign: False and scale: [17.4314, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_18 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_18 INFO:nncf:Set sign: True and scale: [10.1936, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv4_2_ds1]/conv2d_0 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv4_2_ds1]/conv2d_0 INFO:nncf:Set sign: False and scale: [57.7359, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_20 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_20 INFO:nncf:Set sign: True and scale: [22.3863, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv4_3_ds1]/conv2d_0 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv4_3_ds1]/conv2d_0 INFO:nncf:Set sign: False and scale: [43.1886, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_22 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_22 INFO:nncf:Set sign: False and scale: [31.0433, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_24 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_24 INFO:nncf:Set sign: False and scale: [26.4821, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_26 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_26 INFO:nncf:Set sign: True and scale: [14.4154, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv5_2_ds1]/conv2d_0 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv5_2_ds1]/conv2d_0 INFO:nncf:Set sign: False and scale: [126.1439, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_28 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_28 INFO:nncf:Set sign: False and scale: [11.0076, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_12 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_12 INFO:nncf:Set sign: False and scale: [66.5771, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_31 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_31 INFO:nncf:Set sign: False and scale: [41.1410, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_34 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_34 INFO:nncf:Set sign: True and scale: [1.0000, ] for TargetType.OPERATOR_POST_HOOK /nncf_model_input_0 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK /nncf_model_input_0 INFO:nncf:Set sign: False and scale: [28.1410, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_0 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_0 INFO:nncf:Set sign: True and scale: [12.4151, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv1_0_ds1]/conv2d_0 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv1_0_ds1]/conv2d_0 INFO:nncf:Set sign: False and scale: [30.0861, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_2 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_2 INFO:nncf:Set sign: False and scale: [14.0703, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_4 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_4 INFO:nncf:Set sign: True and scale: [16.1223, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_0 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_0 INFO:nncf:Set sign: False and scale: [12.1858, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_6 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_6 INFO:nncf:Set sign: False and scale: [8.8346, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_8 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_8 INFO:nncf:Set sign: False and scale: [10.9855, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_10 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_10 INFO:nncf:Set sign: False and scale: [15.7994, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_13 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_13 INFO:nncf:Set sign: False and scale: [11.1980, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_15 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_15 INFO:nncf:Set sign: False and scale: [12.3363, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_17 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_17 INFO:nncf:Set sign: False and scale: [10.9329, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_19 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_19 INFO:nncf:Set sign: True and scale: [23.5184, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_5 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_5 INFO:nncf:Set sign: False and scale: [10.7701, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_21 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_21 INFO:nncf:Set sign: True and scale: [10.7534, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv5_0_ds1]/conv2d_0 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv5_0_ds1]/conv2d_0 INFO:nncf:Set sign: False and scale: [11.6131, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_23 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_23 INFO:nncf:Set sign: False and scale: [11.6752, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_25 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_25 INFO:nncf:Set sign: True and scale: [19.2094, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_6 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_6 INFO:nncf:Set sign: False and scale: [13.9364, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_27 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_27 INFO:nncf:Set sign: True and scale: [38.4378, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_ds1_det]/conv2d_0 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_ds1_det]/conv2d_0 INFO:nncf:Set sign: False and scale: [43.4875, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_29 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_29 INFO:nncf:Set sign: False and scale: [101.6802, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_30 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_30 INFO:nncf:Set sign: False and scale: [66.5771, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_7 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_7 INFO:nncf:Set sign: False and scale: [69.2373, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_32 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_32 INFO:nncf:Set sign: False and scale: [87.7789, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_33 INFO:nncf:Performing unsigned activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/RELU_33 INFO:nncf:Set sign: True and scale: [7.9520, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv2_0_ds1]/conv2d_0 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv2_0_ds1]/conv2d_0 INFO:nncf:Set sign: True and scale: [9.3574, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv3_0_ds1]/conv2d_0 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv3_0_ds1]/conv2d_0 INFO:nncf:Set sign: True and scale: [13.1900, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_1 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_1 INFO:nncf:Set sign: True and scale: [8.8789, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv4_0_ds1]/conv2d_0 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv4_0_ds1]/conv2d_0 INFO:nncf:Set sign: True and scale: [11.9410, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_3 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_3 INFO:nncf:Set sign: True and scale: [17.4084, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_4 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_4 INFO:nncf:Set sign: True and scale: [11.3707, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv5_1_ds1]/conv2d_0 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/NNCFConv2d[conv5_1_ds1]/conv2d_0 INFO:nncf:Set sign: True and scale: [17.7858, ] for TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_2 INFO:nncf:Performing signed activation quantization for: TargetType.OPERATOR_POST_HOOK MobileNetV2_145M_threechannels/add_2 INFO:nncf:Set sign: True and scale: [4.2162, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv0_0] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv0_0] INFO:nncf:Set sign: True and scale: [6.9671, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv1_0_dw] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv1_0_dw] INFO:nncf:Set sign: True and scale: [1.2043, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv1_0_ds1] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv1_0_ds1] INFO:nncf:Set sign: True and scale: [1.4360, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_0_ds0] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_0_ds0] INFO:nncf:Set sign: True and scale: [9.1817, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_0_dw] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_0_dw] INFO:nncf:Set sign: True and scale: [0.7209, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_0_ds1] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_0_ds1] INFO:nncf:Set sign: True and scale: [0.9147, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_1_ds0] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_1_ds0] INFO:nncf:Set sign: True and scale: [8.2701, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_1_dw] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_1_dw] INFO:nncf:Set sign: True and scale: [0.7909, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_1_ds1] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv2_1_ds1] INFO:nncf:Set sign: True and scale: [0.8546, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_0_ds0] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_0_ds0] INFO:nncf:Set sign: True and scale: [12.7428, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_0_dw] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_0_dw] INFO:nncf:Set sign: True and scale: [0.6166, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_0_ds1] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_0_ds1] INFO:nncf:Set sign: True and scale: [0.5760, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_1_ds0] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_1_ds0] INFO:nncf:Set sign: True and scale: [104.4505, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_1_dw] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_1_dw] INFO:nncf:Set sign: True and scale: [0.4167, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_1_ds1] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_1_ds1] INFO:nncf:Set sign: True and scale: [0.5279, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_2_ds0] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_2_ds0] INFO:nncf:Set sign: True and scale: [7.5171, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_2_dw] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_2_dw] INFO:nncf:Set sign: True and scale: [0.5974, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_2_ds1] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv3_2_ds1] INFO:nncf:Set sign: True and scale: [0.3066, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[deconv_0_ds0] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[deconv_0_ds0] INFO:nncf:Set sign: True and scale: [0.3938, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_0_ds0] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_0_ds0] INFO:nncf:Set sign: True and scale: [4.4389, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_0_dw] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_0_dw] INFO:nncf:Set sign: True and scale: [0.5324, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_0_ds1] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_0_ds1] INFO:nncf:Set sign: True and scale: [0.6500, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_1_ds0] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_1_ds0] INFO:nncf:Set sign: True and scale: [9.0638, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_1_dw] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_1_dw] INFO:nncf:Set sign: True and scale: [0.3157, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_1_ds1] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_1_ds1] INFO:nncf:Set sign: True and scale: [0.4075, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_2_ds0] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_2_ds0] INFO:nncf:Set sign: True and scale: [3.6740, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_2_dw] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_2_dw] INFO:nncf:Set sign: True and scale: [0.4489, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_2_ds1] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_2_ds1] INFO:nncf:Set sign: True and scale: [0.2810, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_3_ds0] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_3_ds0] INFO:nncf:Set sign: True and scale: [52.5925, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_3_dw] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_3_dw] INFO:nncf:Set sign: True and scale: [0.4955, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_3_ds1] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv4_3_ds1] INFO:nncf:Set sign: True and scale: [0.2306, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_0_ds0] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_0_ds0] INFO:nncf:Set sign: True and scale: [3.5721, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_0_dw] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_0_dw] INFO:nncf:Set sign: True and scale: [0.3537, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_0_ds1] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_0_ds1] INFO:nncf:Set sign: True and scale: [0.3996, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_1_ds0] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_1_ds0] INFO:nncf:Set sign: True and scale: [3.9181, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_1_dw] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_1_dw] INFO:nncf:Set sign: True and scale: [0.2571, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_1_ds1] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_1_ds1] INFO:nncf:Set sign: True and scale: [0.5137, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_2_ds0] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_2_ds0] INFO:nncf:Set sign: True and scale: [5.4923, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_2_dw] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_2_dw] INFO:nncf:Set sign: True and scale: [0.4004, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_2_ds1] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv5_2_ds1] INFO:nncf:Set sign: True and scale: [0.3398, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_ds0_det] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_ds0_det] INFO:nncf:Set sign: True and scale: [38.6099, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_dw_det] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_dw_det] INFO:nncf:Set sign: True and scale: [0.5644, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_ds1_det] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_ds1_det] INFO:nncf:Set sign: True and scale: [0.3998, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_ds2_det] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_ds2_det] INFO:nncf:Set sign: True and scale: [4.2655, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_dw2_det] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_dw2_det] INFO:nncf:Set sign: True and scale: [0.5988, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_ds3_det] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv6_0_ds3_det] INFO:nncf:Set sign: True and scale: [0.8722, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv7_0_ds0_det] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv7_0_ds0_det] INFO:nncf:Set sign: True and scale: [4.9182, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv7_0_dw_det] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv7_0_dw_det] INFO:nncf:Set sign: True and scale: [0.6833, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv7_0_ds1_det] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv7_0_ds1_det] INFO:nncf:Set sign: True and scale: [4.3454, ] for TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv_finalout] INFO:nncf:Performing signed weight quantization for: TargetType.OPERATION_WITH_WEIGHTS MobileNetV2_145M_threechannels/NNCFConv2d[conv_finalout] WARNING:nncf:Enabling quantization batch norm adaptation with default parameters. INFO:nncf:BatchNorm statistics adaptation █ | 1 / 11 INFO:nncf:BatchNorm statistics adaptation ██ | 2 / 11 INFO:nncf:BatchNorm statistics adaptation ████ | 3 / 11 INFO:nncf:BatchNorm statistics adaptation █████ | 4 / 11 INFO:nncf:BatchNorm statistics adaptation ███████ | 5 / 11 INFO:nncf:BatchNorm statistics adaptation ████████ | 6 / 11 INFO:nncf:BatchNorm statistics adaptation ██████████ | 7 / 11 INFO:nncf:BatchNorm statistics adaptation ███████████ | 8 / 11 INFO:nncf:BatchNorm statistics adaptation █████████████ | 9 / 11 INFO:nncf:BatchNorm statistics adaptation ██████████████ | 10 / 11 INFO:nncf:BatchNorm statistics adaptation ████████████████ | 11 / 11 WARNING:nncf:Graphviz is not installed - only the .dot model visualization format will be used. Install pygraphviz into your Python environment and graphviz system-wide to enable PNG rendering. /home/wangzhaoming/anaconda3/envs/torch181/lib/python3.7/site-packages/nncf/dynamic_graph/wrappers.py:22: UserWarning: You are using DataParallel, which may cause significant performance issues with dynamic graph building. Consider using distributed training (DistributedDataParallel) instead warnings.warn("You are using DataParallel, which may cause significant performance issues with dynamic graph " step: 0, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 1369.700846, loss_coord: 16.474422, loss_conf: 1353.226400, loss_comp: 0.000000

step: 10, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 1732.981934, loss_coord: 16.648673, loss_conf: 1716.333333, loss_comp: 0.000000

step: 20, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 1993.051921, loss_coord: 18.316709, loss_conf: 1974.735189, loss_comp: 0.000000

step: 30, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 1508.386882, loss_coord: 16.344154, loss_conf: 1492.042806, loss_comp: 0.000000

step: 40, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 1575.248698, loss_coord: 16.095189, loss_conf: 1559.153483, loss_comp: 0.000000

step: 50, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 1570.981608, loss_coord: 14.925048, loss_conf: 1556.056641, loss_comp: 0.000000

step: 60, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 1472.513672, loss_coord: 20.321075, loss_conf: 1452.192546, loss_comp: 0.000000

step: 70, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 1242.894206, loss_coord: 26.722089, loss_conf: 1216.172119, loss_comp: 0.000000

step: 80, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 943.923991, loss_coord: 20.695768, loss_conf: 923.228190, loss_comp: 0.000000

step: 90, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 627.612874, loss_coord: 21.632228, loss_conf: 605.980632, loss_comp: 0.000000

step: 100, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 421.764771, loss_coord: 26.579287, loss_conf: 395.185465, loss_comp: 0.000000

step: 110, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 413.052694, loss_coord: 33.925458, loss_conf: 379.127238, loss_comp: 0.000000

step: 120, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 313.790792, loss_coord: 20.166925, loss_conf: 293.623861, loss_comp: 0.000000

step: 130, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 322.013428, loss_coord: 21.724559, loss_conf: 300.288859, loss_comp: 0.000000

step: 140, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 300.414510, loss_coord: 21.792440, loss_conf: 278.622070, loss_comp: 0.000000

step: 150, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 361.993408, loss_coord: 29.544136, loss_conf: 332.449259, loss_comp: 0.000000

Premature end of JPEG file step: 160, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 265.573466, loss_coord: 19.674287, loss_conf: 245.899170, loss_comp: 0.000000

step: 170, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 285.766520, loss_coord: 22.333300, loss_conf: 263.433228, loss_comp: 0.000000

step: 180, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 342.475382, loss_coord: 32.458460, loss_conf: 310.016927, loss_comp: 0.000000

step: 190, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 306.423319, loss_coord: 27.802508, loss_conf: 278.620809, loss_comp: 0.000000

step: 200, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 289.137451, loss_coord: 25.539330, loss_conf: 263.598124, loss_comp: 0.000000

step: 210, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 314.323140, loss_coord: 27.796000, loss_conf: 286.527140, loss_comp: 0.000000

step: 220, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 359.890706, loss_coord: 35.200714, loss_conf: 324.689982, loss_comp: 0.000000

step: 230, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 251.996379, loss_coord: 21.292262, loss_conf: 230.704122, loss_comp: 0.000000

step: 240, total_step: 2889, epoch: 0, total_epoch: 40

total_loss: 296.313232, loss_coord: 26.210159, loss_conf: 270.103068, loss_comp: 0.000000

ljaljushkin commented 3 years ago

Yes, the estimation of Hessian traces in the HAWQ algorithm has successfully converged, but it seems like the compression ratio is too high for your model - there're too many int4 were chosen and the model struggles to restore quality.

First of all, I'd recommend making sure that fully INT8 quantization doesn't lead to such loss degradation, the pipeline is correct and the model is capable to restore quality.

If this check is OK, then try to gradually increase the compression ratio towards 1.5. BTW, there's a trick to accelerate experiments with different compression ratios. The algorithm dumps estimated Hessian traces to nncf_debug/hawq_dumps/avg_traces_per_layer file in the debug mode enabled by "dump_init_precision_data": true parameter in the config. When you try a different compression ratio, you could avoid the calculation of Hessian traces (the most time-consuming part) again by specifying the path to this file.

...
 "compression": {
        "algorithm": "quantization",
       ...
        "initializer": {
            "precision": {
                "traces_per_layer_path": "path/to/nncf_debug/hawq_dumps/avg_traces_per_layer",
                "compression_ratio": 1.1
                ...
            }
        }
}

We have a plan to automate the procedure of choosing compression ratio for a given maximal accuracy degradation by Accuracy Aware algorithm. But currently, it works only for the filter pruning and sparsity algorithms: https://github.com/openvinotoolkit/nncf/blob/2d5a55b874a2c53acf726caf6c4471fe742aec3e/examples/torch/classification/configs/pruning/resnet50_imagenet_accuracy_aware.json#L28-L29

In addition, it makes sense to use per-channel quantization for weights, because it's more accurate and doesn't add inference overhead on target hardware in comparison with per-tensor one. Asymmetric quantization and per-channel activations also boost accuracy but increase inference latency.

Finally, mixed-precision quantization tends to be trained more efficiently within a stage-quantization pipeline. It enables knowledge distillation, learning rate annealing and allows to tune quantization of activations first without weights to not drop accuracy a lot on the first stages of tuning. https://github.com/openvinotoolkit/nncf/blob/2d5a55b874a2c53acf726caf6c4471fe742aec3e/examples/torch/classification/configs/mixed_precision/mobilenet_v2_imagenet_mixed_int_manual_staged.json#L158-L167

betterhalfwzm commented 3 years ago

Yes, the estimation of Hessian traces in the HAWQ algorithm has successfully converged, but it seems like the compression ratio is too high for your model - there're too many int4 were chosen and the model struggles to restore quality.

First of all, I'd recommend making sure that fully INT8 quantization doesn't lead to such loss degradation, the pipeline is correct and the model is capable to restore quality.

If this check is OK, then try to gradually increase the compression ratio towards 1.5. BTW, there's a trick to accelerate experiments with different compression ratios. The algorithm dumps estimated Hessian traces to nncf_debug/hawq_dumps/avg_traces_per_layer file in the debug mode enabled by "dump_init_precision_data": true parameter in the config. When you try a different compression ratio, you could avoid the calculation of Hessian traces (the most time-consuming part) again by specifying the path to this file.
...
 "compression": {
        "algorithm": "quantization",
       ...
        "initializer": {
            "precision": {
                "traces_per_layer_path": "path/to/nncf_debug/hawq_dumps/avg_traces_per_layer",
                "compression_ratio": 1.1
                ...
            }
        }
} 
We have a plan to automate the procedure of choosing compression ratio for a given maximal accuracy degradation by Accuracy Aware algorithm. But currently, it works only for the filter pruning and sparsity algorithms:

https://github.com/openvinotoolkit/nncf/blob/2d5a55b874a2c53acf726caf6c4471fe742aec3e/examples/torch/classification/configs/pruning/resnet50_imagenet_accuracy_aware.json#L28-L29

In addition, it makes sense to use per-channel quantization for weights, because it's more accurate and doesn't add inference overhead on target hardware in comparison with per-tensor one. Asymmetric quantization and per-channel activations also boost accuracy but increase inference latency.

Finally, mixed-precision quantization tends to be trained more efficiently within a stage-quantization pipeline. It enables knowledge distillation, learning rate annealing and allows to tune quantization of activations first without weights to not drop accuracy a lot on the first stages of tuning. https://github.com/openvinotoolkit/nncf/blob/2d5a55b874a2c53acf726caf6c4471fe742aec3e/examples/torch/classification/configs/mixed_precision/mobilenet_v2_imagenet_mixed_int_manual_staged.json#L158-L167

@ljaljushkin Thank you very much for your work and suggestions. I will try them one by one.

openvinotoolkit / nncf

train mix bit[4,8],but get the model look like int8 #785