search

google / qkeras

QKeras: a quantization deep learning library for Tensorflow Keras
Apache License 2.0
532 stars 102 forks source link

Quantize the Model without retraining ? #81

Closed HenningCode closed 2 years ago

HenningCode commented 2 years ago

Hello Guys,

is it possible to quantize a model with AutoQKeras without the need of retraining? I know this wont yield the best results I am just curios of it is possible.

Maybe if the transfer_weights = true parameter is set?

danielemoro commented 2 years ago

Hi. You don't necessarily need AutoQKeras to quantize a model without training. You can simply use the model_quantize function in https://github.com/google/qkeras/blob/master/qkeras/utils.py#L376 to quantize an existing Keras model, and then you can run that model without training. Of course, I would not recommend that since you will get poor results, but it is possible to do.

HenningCode commented 2 years ago

Today I finally got around to test the model_quantize and the results were as you predicted pretty pretty poor. I only got a accuracy of around 10% with the mnist dataset. Is there a possibility to get a higher accuracy or is this straight up impossible ?

danielemoro commented 2 years ago

There is no existing algorithm in QKeras to perform post-training quantization with high accuracy. We highly recommend quantization-aware training instead. In the future we may add post-training quantization algorithms in QKeras.