search

kendryte / nncase

Open deep learning compiler stack for Kendryte AI accelerators ✨
Apache License 2.0
734 stars 181 forks source link

int8 quantization don't reduce the size of the model #553

Closed Charlie839242 closed 2 years ago

Charlie839242 commented 2 years ago

Describe the bug I used the following compile options to convert the onnx model to kmodel. However, the size of the two models are the same so I am wondering whether the model is quantized successfully.

  compile_options = nncase.CompileOptions()
  compile_options.target = target
  compile_options.quant_type = 'int8'
  compile_options.w_quant_type = 'int8'
  compile_options.input_type = 'int8'
  compile_options.output_type = 'int8'
  compile_options.dump_ir = True
  compile_options.dump_asm = True
  compile_options.dump_dir = 'tmp'

Expected behavior I expect the kmodel size to be much smaller.

Environment (please complete the following information):

sunnycase commented 2 years ago

Please upload your origin model.

Charlie839242 commented 2 years ago

Thanks. The model is in the following link:

https://github.com/Charlie839242/line-seg-on-jeston-k210/tree/main/model
curioyang commented 2 years ago

@Charlie839242 https://github.com/kendryte/nncase/blob/master/docs/USAGE_EN.md#compile-uint8-model-for-tflite You need to set ptq_options

curioyang commented 2 years ago

@Charlie839242 If any questions, reopen the issue.