search

VeriSilicon / acuitylite

Acuitylite is an end-to-end neural network deployment tool
https://verisilicon.github.io/acuitylite
MIT License
16 stars 5 forks source link

TFLiteExporter does not export quantized model #13

Closed solmersa closed 1 month ago

solmersa commented 1 year ago

Hi, I was testing importing a TFLite model fp16, and then export it again as TFLite using the TFLiteExporter.

Looks like the exporter is using fp32 for the export as the output model is twice the size and I don't see any int8 / uint8 weights.

Does the exporter uses the quantized weights?

lileiigithub commented 1 year ago

Hi, Maybe you need to quantize the model before exporting it such as : model.quantize(data_list, quantizer=QuantizeType.pcq_symi8, iteration=1) in https://github.com/VeriSilicon/acuitylite/blob/main/demo/lenet_caffe_test.py

solmersa commented 1 year ago

Thanks for the reply. I was expecting it would preserve the data type, somewhat round tripping the original format but it does not.

Also when doing quantization there is an error when it tries to remove the dead channels and it aborts the process.

geneyoung commented 6 months ago

Generally, FP16 TFLite model will be converted into AcuityLite format and save data as float32. If you want to export a quantized format, you need do quantization and then export. Seems there are some bugs when doing dead channel removing, can you provide you sample model so that we can analysis more?