Mask RCNN on TensorFlow Lite

[JaviBonilla]

Has anyone used Mask RCNN on TensorFlow Lite? I managed to create a tflite file from my network, but there are some TensorFlow Lite unsupported operators used in Mask RCNN: ResizeNearestNeighbor, Stack, and TensorFlowShape.

Some of the operators in the model are not supported by the standard TensorFlow Lite runtime. If you have a custom implementation for them you can disable this error with --allow_custom_ops. Here is a list of operators for which you will need custom implementations: ResizeNearestNeighbor, Stack, TensorFlowShape.

Has anyone managed to implement such operators? There is some information over here about how to do it, but I wanted to ask in case that someone already implemented them. https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/g3doc/custom_operators.md.

[ghost]

@JaviBonilla did you solve this?

[JaviBonilla]

Hi @CoolJack11,

No, I'm still trying to figure it out. I am testing with the Mask RCNN Inception v2 COCO from TensoFlow detection model zoo, but there are still some issues. Have a look at https://github.com/tensorflow/tensorflow/issues/19293

[JonathanLehner]

any updates on this?

[JaviBonilla]

@JonathanLehner No luck yet, but the TensorFlow guys are looking at the Mask RCNN TensorFlow Lite conversion failure at https://github.com/tensorflow/tensorflow/issues/19293

[jps892]

@JaviBonilla any update on this? please if you have any let us know

[JaviBonilla]

@jps892 No updates sorry. Sure, I will let you know if there are any news.

[lucasjinreal]

Pls make some speed test after u migrate pb model into tflite. We'd like to run it on a edge device.

[PINTO0309]

I tried to generate tflite. It includes custom operations. Mask RCNN InceptionV2 https://github.com/PINTO0309/PINTO_model_zoo

[xavier-faromatics]

Any updates or alternative models?

[bmabir17]

This is how i was able to convert it https://gist.github.com/bmabir17/754a6e0450ec4fd5e25e462af949cde6

did any of you tried to run inference on it using python interpreter or android?

[ValentinRicher]

This is how i was able to convert it https://gist.github.com/bmabir17/754a6e0450ec4fd5e25e462af949cde6

did any of you tried to run inference on it using python interpreter or android?

Hello, I used your code to generate the .tflite model and it worked, thank you very much. However, when I tried to used the lite model with the Python interpreter I had a "segmentation fault" problem.

Here is my code :

import numpy as np
import tensorflow as tf

# Load TFLite model and allocate tensors.
interpreter = tf.lite.Interpreter(model_path="mask_rcnn_coco.tflite")
interpreter.allocate_tensors()

# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

# Test model on random input data.
input_shape = input_details[0]['shape']
input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32)
interpreter.set_tensor(input_details[0]['index'], input_data)

interpreter.invoke()

# The function `get_tensor()` returns a copy of the tensor data.
# Use `tensor()` in order to get a pointer to the tensor.
output_data = interpreter.get_tensor(output_details[0]['index'])
print(output_data)

The error is "Segmentation fault: 11" occurring at interpreter.invoke() I don't know if you faced the same issue when trying to use it with python, android or iOS

[bmabir17]

It seems the tflite python interpreter does not support the select_tf_ops. This is under active development. So i am now trying to infer the model using Android interpreter.

[sayakpaul]

I am now getting the following:

RuntimeError: tensorflow/lite/kernels/gather.cc:80 0 <= axis && axis < NumDimensions(input) was not true.Node number 222 (GATHER) failed to prepare.

I am using tf-nightly (version: 2.3.0-dev20200528). Used the above code snippet and the TF Lite model converted using your awesome gist @bmabir17.

[bmabir17]

@sayakpaul I have faced the same error. And have created an issue in tensorflow repo, but still have not found any solution for it

[sayakpaul]

Thanks for letting me know :)

[ulhaqi12]

Hi, Has anyone succeeded? I have tried converting in using code on this https://gist.github.com/bmabir17/754a6e0450ec4fd5e25e462af949cde6. I obtained .tflite file but it is not working with tflite interpreter in python. giving some errors. Anyone, who able to create a working tflite model of Mask RCNN? Best, Ikram

[bmabir17]

I was going through the converted model and found that tflite converted model has some difference compared with the original model.pb(saved before conversion to tflite). It seems the range op conversion has some problem. And this seems to be the input for the expand dims. Tensorflow Model(original) Tflite Model(converted) Can anyone help why is this happening? P.S the model conversion showed no error. My conversion code

[bmabir17]

@sayakpaul It seems if you convert the model using toco converter from tensorflow 1.15.0 the problem goes away. But now i get the a different set of errors, because of the problem explained on my previous comment

[oholimoli]

Is now possible to convert RNNs with masks to tflite?

[madrugou]

Hello, I am trying to convert mask-rcnn to tflite for later running inference in python and c++.

What worked for me in the conversion was to follow this article: Convert Mask R-CNN model to TFLite with Tensorflow 2.3

And for python inference I had to install

pip install tf-nightly

My code:

from loguru import logger
from time import time
import tensorflow as tf
import numpy as np

logger.info('Load the model')
model = tf.lite.Interpreter(model_path='./models/model.tflite')
model.allocate_tensors()

input_details = model.get_input_details()
output_details = model.get_output_details()
input_data = np.array(np.random.random_sample(input_details[0]['shape']), dtype=np.float32)

logger.info('Predict input data')
model.set_tensor(input_details[0]['index'], input_data)

start_time = time()
model.invoke()
print(time()-start_time)

output_data = model.get_tensor(output_details[0]['index'])
print(output_data)

logger.success('Finish.')

The output of the code above:

...
2021-05-04 11:42:50.564 | INFO     | __main__:<module>:16 - Predict input data
30.77747941017151
[[[0. 0. 0. 0.]
  [0. 0. 0. 0.]
  [0. 0. 0. 0.]
  ...
  [0. 0. 0. 0.]
  [0. 0. 0. 0.]
  [0. 0. 0. 0.]]]
2021-05-04 11:43:21.348 | SUCCESS  | __main__:<module>:27 - Finish.

[sauravsolanki]

Hello, I am trying to convert mask-rcnn to tflite for later running inference in python and c++.

What worked for me in the conversion was to follow this article: Convert Mask R-CNN model to TFLite with Tensorflow 2.3

And for python inference I had to install

pip install tf-nightly

My code:

from loguru import logger
from time import time
import tensorflow as tf
import numpy as np

logger.info('Load the model')
model = tf.lite.Interpreter(model_path='./models/model.tflite')
model.allocate_tensors()

input_details = model.get_input_details()
output_details = model.get_output_details()
input_data = np.array(np.random.random_sample(input_details[0]['shape']), dtype=np.float32)

logger.info('Predict input data')
model.set_tensor(input_details[0]['index'], input_data)

start_time = time()
model.invoke()
print(time()-start_time)

output_data = model.get_tensor(output_details[0]['index'])
print(output_data)

logger.success('Finish.')

The output of the code above:

...
2021-05-04 11:42:50.564 | INFO     | __main__:<module>:16 - Predict input data
30.77747941017151
[[[0. 0. 0. 0.]
  [0. 0. 0. 0.]
  [0. 0. 0. 0.]
  ...
  [0. 0. 0. 0.]
  [0. 0. 0. 0.]
  [0. 0. 0. 0.]]]
2021-05-04 11:43:21.348 | SUCCESS  | __main__:<module>:27 - Finish.

Hi @GabrielMLB , Can you please share the environment here? I could not run the Tflite Model. It is taking like forever and crashing my kernel. One colab link with requirement.txt will be great help to many.

[madrugou]

@sauravsolanki actually the execution(inference) for me took about 30 seconds and I still don't get the expected result (when the kernel doesn't crash), I'll share the Colab link that I managed to convert to tflite, but the inference is not good, it takes too long to return something, usually it crashes in model.invoke(), I still can't solve this problem. Colab Link

[RITIK-12]

Hey @GabrielMLB will you please share the link for the code where you have taken the inference on the tflite model. Thanks