General questions about the quantisation process

[Petros626]

Issue Type

Feature Request, Documentation Feature Request

OS

Windows

OS architecture

x86_64

Programming Language

Python

Framework

TensorFlow, TensorFlowLite

Model name and Weights/Checkpoints URL

--

Description

Hello @PINTO0309

first a big thank you for your effort, the repo of the TensorFlow OD API definitely needs an update. 👍 I assume you only use quantization after training (representative dataset, functions in scripts etc.), as quantization before training (only known from TF OD API "ssd_mobilenetv2_quantized") would be more complex to implement. My questions would be:

1. the scripts for quantisation mostly contain known datasets for quantisation like COCO. Is a simple adaptation to my dataset (images in the folder) easily possible?

2. will you show in the future how to create real fixed point integer model? The TensorFlow team only provides the models ready to use. They describe the process of quantisation pretty well with overwriting both graphs (training + eval )(https://github.com/tensorflow/tensorflow/tree/r1.15/tensorflow/contrib/quantize).

Here I would like to pick up on the following:

"The previously demonstrated after-rewrite eval graph only simulates quantization. To generate real fixed-point computations from a trained quantization model, convert it to a fixed-point kernel. TensorFlow Lite supports this conversion from the graph resulting from create_eval_graph."

Here I am not quite clear about the trained quantized model. Is it trained with float32 and converted to int8 in the course or is int8 used from the beginning? As I mentioned, it's a bit of a mystery since TensorFlow only publishes quantized models, but doesn't mention how to create or train your own.

I would appreciate a detailed answer to enlighten more users

Relevant Log Output

--

URL or source code for simple inference testing code

--

[PINTO0309]

Here is a good place to ask the question. https://discuss.tensorflow.org/

[Petros626]

I was actually hoping to get some good information from you as I haven't really received any feedback on this forum so far :joy::joy: but okay, thanks for taking a look.