search

openvinotoolkit / openvino

OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
https://docs.openvino.ai
Apache License 2.0
7.36k stars 2.3k forks source link

Failed to quantize FP32 model to INT8 on DL Workbench #471

Closed yushuinanrong closed 4 years ago

yushuinanrong commented 4 years ago

Hi,

Thanks for sharing this great platform! I'm trying to quantize my pretrained image classification model (originally trained in Pytorch, the model has been transferred to .onnx format) using OpenVINO DL Workbench. I managed to import my model, validation dataset and run the evaluation. However, I cannot quantize the model to INT8 when I did the following "Optimizing" --> "INT8" --> "Optimizing method: default" or "Optimizing method: AccuracyAware". The error message I got says: "Calibration tool failed in stage: Start computing statistics for algorithm: MinMax Quantization, FastBiasCorrection(2)"

Any hint is greatly appreciated.

Screen Shot 2020-04-27 at 1 52 46 PM

P.S. I also tried to quantize models in different formats downloaded from the open model zoo. But I always got the same error.

Some environment info: OS: Mac OS 10.15.3 OpenVINO: 2020.2.120 CPU: Intel i9-9880H Docker: 17.03.1-ce-rc1-mac3 (15924)

AlexKoff88 commented 4 years ago

Hi @yushuinanrong ,

Could you please provide all the logs including the whole trace? Unfortunately, this is not possible to understand what happened from the image above.

yushuinanrong commented 4 years ago

Hi @AlexKoff88， I cannot find the log file. Do you know where it could be?

AlexKoff88 commented 4 years ago

It seems that the logs are stored for a short period of time in the tool. Are you able to run the experiment again? We could provide guidelines on what can be done to capture the artifacts for analysis.

yushuinanrong commented 4 years ago

Hi @AlexKoff88 , Unfortunately my project has a tight deadline so I gave up trying at some point and switched to Pytorch quantization module.

AlexKoff88 commented 4 years ago

Hi @yushuinanrong,

Just in case if you would like to get back to OpenVINO inference I would suggest maybe directly using the command-line tools: https://docs.openvinotoolkit.org/latest/_docs_LowPrecisionOptimizationGuide.html.

BTW, what kind of model are you going to speed up and what is the target platform? Is it Core i9-9880H?

yushuinanrong commented 4 years ago

@AlexKoff88 Thanks for your suggestion. I spent a bit of time on the command-line tools but got no luck either. I would try it again for future projects. I did all OpenVINO experiments on my mac which has a I9 cpu, but the algorithm will be deployed in a server with Intel Xeon CPUs.

AlexKoff88 commented 4 years ago

@yushuinanrong, what was wrong this the command-line tools. Did they crash? or you got poor results. If you can share your configuration file for the post-training tool we could try to help you.