Support TFLite_Detection_PostProcess Op

[aminbenarieb]

🐞Describe the bug

I am trying to convert tensorflow SSD MobilenetV2 model (frozen graph) to coreml model using coremltools.converters.tensorflow.convert function and getting error Op type not registered 'TFLite_Detection_PostProcess' in binary.

As far as I understand, this Op is not supported in CoreML. Here is its C++ implementation: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/kernels/detection_postprocess.cc

This blocks me from using CoreML for my detection problem. Any thoughts on:

1. When the Op will be supported?
2. Is there a quick workaround? Implementing and maintaining that code in Swift is too hard

Trace

Traceback (most recent call last):
  File "coreml.py", line 144, in <module>
    main()
  File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "coreml.py", line 129, in main
    mlmodel_path
  File "coreml.py", line 44, in tf_convert_to_coreml
    the_graph = load_graph(tf_model_path)
  File "coreml.py", line 62, in load_graph
    tfc.import_graph_def(original_gdef, name='')
  File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 427, in import_graph_def
    graph._c_graph, serialized, options)  # pylint: disable=protected-access
tensorflow.python.framework.errors_impl.NotFoundError: Op type not registered 'TFLite_Detection_PostProcess' in binary running on Amins-MBP.Dlink. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.
```'TFLite_Detection_PostProcess' in binary running on Amins-MBP.Dlink. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.

My Code

import coremltools

def convert_to_coreml(
    model_path,
    prototxt_path,
    class_labels_path,
    mlmodel_path
    ):
    print(model_path, prototxt_path)
    coreml_model = coremltools.converters.tensorflow.convert(
        model_path, predicted_feature_name=class_labels_path
    )

    coreml_model.save(mlmodel_path)

System environment:

• coreml version: 3.3
• tensorflow versions: 2.1 (used for TF frozen graph):
• macOS version 10.15):
• How you install python (virtualenv):
• python 3.7.6:

[aminbenarieb]

Also, we managed to slice this op from our graph and successfully run CoreML, but, this quite an ambiguous and error-prone how to interpret inference results. So, still, we need some workflow for this missed op.

[aminbenarieb]

@1duo can I mention you here? I just realized that a more correct label is a function request, not an error. Should I provide more details, it might be helpful if I provided a model to reproduce what we are facing?

[al-muammar]

+1, we are blocked by this issue as well. AFAIK, this layer is a standard one in detection, and it is pretty wide-spread to run detection on mobile devices.