Image Preprocessing Operators

[jackm321]

Create Glow IR nodes and instructions for representing common pre/post processing tasks especially for image classification workloads for example:

• YUV to RGB
• scaling
• cropping
• pixel to tensor conversions

[mciprian13]

I think this is very useful. For most applications the bundle is not self-sufficient since it requires some pre-processing of the image and some post-processing. By adding these in the graph the AOT bundles might become self-sufficient. I would also add to your list:

• PNG/JPEG decompressor (it sounds kind of far fetched but this is very useful for embedded since there are camera devices which can generate images directly in PNG/JPEG, images thus travel fast through the memory buses - TensorFlowLite has something like this)
• NMS (Non-Max Suppresion) for SSD models - post-processing used to filter out the relevant output boxes
• TopK (already exists)

Apart from having the Nodes implemented and using them from the C++ API, I would be interested also in how should be make this functionality available for the front-end tools as well (e.g. model-compiler, image-classifier):

• should these operators be added during the import phase of ONNX/Caffe2 models?
• how to specify through CLI parameters that I want to add in the graph (for example) a PNG decompressor & Scale node for the input placeholder and a TopK node for the output placeholder?

I would imagine the most common case would be this: I have a standard (say) model like MobileNet in ONNX format, and I want to compile it into a bundle but I also want to add the pre-processing/post-processing nodes into the graph such that I don`t have to implement that in the application.

What do you think?