Error runing quantization_speedup.py

[Raychen0617]

Describe the issue: When running quantization_speedup.py in the tutorial file (I did not change anything), I got an error as below.

[2022-07-20 01:41:46] Model state_dict saved to ./log/mnist_model.pth
[2022-07-20 01:41:46] Mask dict saved to ./log/mnist_calibration.pth
[07/20/2022-01:41:49] [TRT] [W] DynamicRange(min: -0.424213, max: 2.82149). Dynamic range should be symmetric for better accuracy.
Traceback (most recent call last):
  File "quantization_speedup.py", line 114, in <module>
    engine.compress()
  File "/opt/conda/lib/python3.7/site-packages/nni/compression/pytorch/quantization_speedup/integrated_tensorrt.py", line 298, in compress
    context = self._tensorrt_build_withoutcalib(self.onnx_path)
  File "/opt/conda/lib/python3.7/site-packages/nni/compression/pytorch/quantization_speedup/integrated_tensorrt.py", line 348, in _tensorrt_build_withoutcalib
    engine = build_engine(onnx_path, self.onnx_config, self.extra_layer_bits, self.strict_datatype)
  File "/opt/conda/lib/python3.7/site-packages/nni/compression/pytorch/quantization_speedup/integrated_tensorrt.py", line 198, in build_engine
    handle_gemm(network, i, config)
  File "/opt/conda/lib/python3.7/site-packages/nni/compression/pytorch/quantization_speedup/integrated_tensorrt.py", line 82, in handle_gemm
    pre_in_tensor.dynamic_range = (tracked_min_input, tracked_max_input)
AttributeError: 'NoneType' object has no attribute 'dynamic_range'

Environment:

• NNI version: 2.8
• Training service (local|remote|pai|aml|etc): local
• Client OS: Ubuntu20.04
• Server OS (for remote mode only):
• Python version: 3.7.10
• PyTorch/TensorFlow version: 1.9.0
• Is conda/virtualenv/venv used?: no
• Is running in Docker?: yes
• GPU: 3090
• cuda: 11.1
• Nvidia-tensorrt: 8.4.1.5

Configuration:

• Experiment config (remember to remove secrets!):
• Search space:

Log message:

• nnimanager.log:
• dispatcher.log:
• nnictl stdout and stderr:

How to reproduce it?:

[QuanluZhang]

@Raychen0617 , thanks for reporting this issue, which tutorial did you run?

[Raychen0617]

This one https://github.com/microsoft/nni/blob/e8c78bbab03fa651563e21cdf004220f29d9560a/examples/tutorials/quantization_speedup.py