Converting Keras CV Stable Diffusion models to tflite
environments
TensorFlow: 2.12.1
keras: 2.12.0
keras-cv: 0.4.2
Note that different version of TensorFlow and Keras could generate
incompatible tflite files.
Conversion scripts
fp32
Tested with TensorFlow master branch (supposedly, using tf-nightly should be fine)
- update group normalization, if you run into related problems, https://github.com/keras-team/keras-cv/pull/1035
- text encoder and decoder: Converting the text_encoder and decoder is trivial.
- diffusion model: Converting the diffusion model needs extra effort. The model weights of the diffusion models is about 3.4 GiB, which is much larger than file size limit (2 GiB) of flatbuffer, the format TFLite used for its models. Surely, it's possible to modify flatbuffer to use 64-bit offset so that we can overcome the 2 GiB limit, but that will result in imcompatible files.
full-integer quantization
It's also possible to generate Quantized int8 models with Post-Training Quantization (PTQ). I don't know how to generate representative datasets, but as a proof of concept, I wrote a script that uses only one sample input as the dataset :-)
dynamic range quantization
Conversion script
Testing/Verifying converted tflite models
Borrowing some code from Keras CV implementation, we can do end-to-end test of converted TFLite models
test image encoder model with inpainting
Converted models
I put tflite models I converted to HuggingFace