How to use yolo_tiny to make INT8 Quantization?

[su26225]

I make some changes in yolov4_416x416_qtz.json and accuracy_checker\adapters\yolo.py as follows: "type": "yolo_v3", "anchors": "10.0, 14.0, 23.0, 27.0, 37.0, 58.0, 81.0, 82.0, 135.0, 169.0, 344.0, 319.0", "classes": 2, "coords": 4, "num": 6, "threshold": 0.001, "anchor_masks": [ [3, 4, 5], [1, 2, 3]], "outputs": ["detector/yolo-v4/Conv_1/BiasAdd/YoloRegion", "detector/yolo-v4/Conv_9/BiasAdd/YoloRegion"] and class YoloV3Adapter(Adapter): 'anchors': default='tiny_yolo_v3', 'cells': ListField default=[13, 26],

But I got the error : Traceback (most recent call last): File "D:\python3.6.5\Scripts\pot-script.py", line 33, in sys.exit(load_entry_point('pot==1.0', 'console_scripts', 'pot')()) File "C:\Intel\openvino_2021.3.394\deployment_tools\tools\post_training_optimization_toolkit\app\run.py", line 37, in main app(sys.argv[1:]) File "C:\Intel\openvino_2021.3.394\deployment_tools\tools\post_training_optimization_toolkit\app\run.py", line 56, in app metrics = optimize(config) File "C:\Intel\openvino_2021.3.394\deployment_tools\tools\post_training_optimization_toolkit\app\run.py", line 123, in optimize compressed_model = pipeline.run(model) File "C:\Intel\openvino_2021.3.394\deployment_tools\tools\post_training_optimization_toolkit\compression\pipeline\pipeline.py", line 57, in run result = self.collect_statistics_and_run(model, current_algo_seq) File "C:\Intel\openvino_2021.3.394\deployment_tools\tools\post_training_optimization_toolkit\compression\pipeline\pipeline.py", line 67, in collect_statistics_and_run model = algo.run(model) File "C:\Intel\openvino_2021.3.394\deployment_tools\tools\post_training_optimization_toolkit\compression\algorithms\quantization\default\algorithm.py", line 93, in run self.algorithms[1].algo_collector.compute_statistics(model) File "C:\Intel\openvino_2021.3.394\deployment_tools\tools\post_training_optimization_toolkit\compression\statistics\collector.py", line 73, in computestatistics , stats_ = self._engine.predict(combined_stats, sampler) File "C:\Intel\openvino_2021.3.394\deployment_tools\tools\post_training_optimization_toolkit\compression\engines\ac_engine.py", line 169, in predict stdout_redirect(self._model_evaluator.process_dataset_async, *args) File "C:\Intel\openvino_2021.3.394\deployment_tools\tools\post_training_optimization_toolkit\compression\utils\logger.py", line 132, in stdout_redirect res = fn(args, **kwargs) File "C:\Intel\openvino_2021.3.394\deployment_tools\tools\post_training_optimization_toolkit\libs\open_model_zoo\tools\accuracy_checker\accuracy_checker\evaluators\quantization_model_evaluator.py", line 153, in process_dataset_async batch_raw_predictions, batch_identifiers, batch_meta File "C:\Intel\openvino_2021.3.394\deployment_tools\tools\post_training_optimization_toolkit\libs\open_model_zoo\tools\accuracy_checker\accuracy_checker\evaluators\quantization_model_evaluator.py", line 99, in _process_ready_predictions return self.adapter.process(batch_raw_predictions, batch_identifiers, batch_meta) File "C:\Intel\openvino_2021.3.394\deployment_tools\tools\post_training_optimization_toolkit\libs\open_model_zoo\tools\accuracy_checker\accuracy_checker\adapters\yolo.py", line 393, in process predictions[b].append(raw_outputs[blob][b]) KeyError: 'detector/yolo-v4/Conv_1/BiasAdd/YoloRegion'

[TNTWEN]

just change “anchors” "anchor_masks" "outputs"

for "outputs" ,you could use netron to locate IRmodel's yoloregion

[su26225]

just change “anchors” "anchor_masks" "outputs"

for "outputs" ,you could use netron to locate IRmodel's yoloregion

Thanks . I have made the Quantization from f32 to int8 successfully .

And now I try to make the Quantization from f32 to f16 , I have changed the parameters in yolov4_416x416_qtz.json as follow: "weights": { // Weights quantization parameters used by MinMaxAlgorithm "bits": 16, // Bit-width, default is 8 "mode": "symmetric", // Quantization mode, default is "symmetric" "level_low": 0, // Minimum level in the integer range in which we quantize to, default is 0 for unsigned range, -2^(bit-1) - for signed "level_high": 65535, // Maximum level in the integer range in which we quantize to, default is 2^bits-1 for unsigned range, 2^(bit-1)-1 - for signed "granularity": "perchannel", // Quantization scale granularity: ["pertensor" (default), "perchannel"] "activations": { "bits": 16, // Number of quantization bits "mode": "symmetric", // Quantization mode "granularity": "pertensor", // Granularity: one scale for output tensor But the size of .bin has not changed . I get the warning : WARNING:compression.algorithms.quantization.fake_quantize_configuration:Fake quantize node detector/yolo-v4-tiny/Conv_14/Conv2D/fq_weights_1 does not support configuration from tool config file (mismatch with hardware config) At the same time , I got a note from the documentation ,that is "It worth noting that changing the quantization scheme may lead to inability to infer such mode on the existing HW." https://docs.openvinotoolkit.org/latest/pot_compression_algorithms_quantization_default_README.html Can this Quantization continue? Thanks!

[TNTWEN]

FP32 ->FP16 Just follow FAQ ponit 10 There is no need to use pot