intel / neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime
https://intel.github.io/neural-compressor/
Apache License 2.0
2.22k stars 255 forks source link

How to set yaml conf When ptq A tensorflow2.x saved_model? #62

Closed johnsGuo closed 2 years ago

johnsGuo commented 2 years ago

I have a saved_model want to use NIC's PTQ, When I use examples/tensorflow/image_recognition/SavedModel/quantization/ptq to test it, The error occur:

2022-03-23 17:08:39 [INFO] Generate a fake evaluation function.
Traceback (most recent call last):
  File "main.py", line 59, in <module>
    evaluate_opt_graph.run()
  File "main.py", line 48, in run
    q_model = quantizer.fit()
  File "/root/anaconda3/envs/tf2/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in __call__
    return super(Quantization, self).__call__()
  File "/root/anaconda3/envs/tf2/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 214, in __call__
    self.pre_process()
  File "/root/anaconda3/envs/tf2/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 121, in pre_process
    self._create_calib_dataloader(cfg)
  File "/root/anaconda3/envs/tf2/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 112, in _create_calib_dataloader
    self._calib_dataloader = create_dataloader(self.framework, calib_dataloader_cfg)
  File "/root/anaconda3/envs/tf2/lib/python3.7/site-packages/neural_compressor/utils/create_obj_from_config.py", line 96, in create_dataloader
    copy.deepcopy(dataloader_cfg['filter']),)
  File "/root/anaconda3/envs/tf2/lib/python3.7/site-packages/neural_compressor/utils/create_obj_from_config.py", line 79, in create_dataset
    transform=preprocess, filter=filter)
  File "/root/anaconda3/envs/tf2/lib/python3.7/site-packages/neural_compressor/experimental/data/datasets/dataset.py", line 764, in __new__
    raise ValueError('Found no files in --root matching: {}'.format(glob_pattern))
ValueError: Found no files in --root matching: ./data/*-*-of-*

my yaml conf:

model:                                               # mandatory. used to specify model specific information.
  name: origin_model
  framework: tensorflow                              # mandatory. supported values are tensorflow, pytorch, pytorch_ipex, onnxrt_integer, onnxrt_qlinear or mxnet; allow new framework backend extension.
  inputs: dense_input, sparse_ids_input, sparse_wgt_input, seq_50_input
  outputs: dense

quantization:                                        # optional. tuning constraints on model-wise for advance user to reduce tuning space.
  calibration:
    sampling_size: 20000                              # optional. default value is 100. used to set how many samples should be used in calibration.
    dataloader:
      batch_size: 10
      dataset:
         TFRecordDataset:
           root: ./data
  model_wise:                                        # optional. tuning constraints on model-wise for advance user to reduce tuning space.
    activation:
      algorithm: minmax
  op_wise: {
    'import/dnn/hiddenlayer_0/MatMul': {
      'activation':  {'dtype': ['uint8'], 'algorithm': ['minmax'], 'scheme':['asym']},
    }
  }

tuning:
  accuracy_criterion:
    relative:  0.01                                  # optional. default value is relative, other value is absolute. this example allows relative accuracy loss: 1%.
  exit_policy:
    timeout: 0                                       # optional. tuning timeout (seconds). default value is 0 which means early stop. combine with max_trials field to decide when to exit.
    max_trials: 100                                  # optional. max tune times. default value is 100. combine with timeout field to decide when to exit.
  random_seed: 9527                                  # optional. random seed for deterministic tuning.
image
johnsGuo commented 2 years ago

I have fix it

ftian1 commented 2 years ago

the yaml file need to be updated accordingly. as you have fixed this issue, I will close it