Can't Convert tensorflow saved_model to frozen inference graph

[Deep-learning-practitioner]

System information (version)

Using Google colab TF version = 2.2

I have fine tuned an SSD MobileNet V2 model in tensorflow Objecet Detection API 2. Now I'm trying to convert the saved_model to frozen inference graph so I can use it in OpenCV DNN module but I'm really confused as how to proceed.

I Have been following this tutorial, but how to modify this for mobilenet v2 trained with TFOD API v2.

The saved model I'm trying to convert can be loaded from zoo via this code: You can run this in colab

!wget 'http://download.tensorflow.org/models/object_detection/tf2/20200711/ssd_mobilenet_v2_320x320_coco17_tpu-8.tar.gz'
!tar -xf ssd_mobilenet_v2_320x320_coco17_tpu-8.tar.gz
model_path = 'ssd_mobilenet_v2_320x320_coco17_tpu-8/saved_model/saved_model.pb'

import tensorflow as tf
loaded = tf.saved_model.load('ssd_mobilenet_v2_320x320_coco17_tpu-8/saved_model') 

Now how can I convert this model to frozen inference graph?

While following the above tutorial and running this code:

full_model = tf.function(lambda x: model(x))

full_model = full_model.get_concrete_function(tf.TensorSpec(loaded.inputs[0].shape, loaded.inputs[0].dtype))

I'm getting this error:

      1 full_model = tf.function(lambda x: model(x))
----> 2 full_model = full_model.get_concrete_function(tf.TensorSpec(loaded.inputs[0].shape, loaded.inputs[0].dtype))

AttributeError: '_UserObject' object has no attribute 'inputs'

Thank you, any help would be appreciated

[sglvladi]

@Deep-learning-practitioner have a look here for an example on using the saved model format.

As noted here frozen graphs are not supported in TF2.

Hope this helps.

[saikumarchalla]

@Deep-learning-practitioner Could you please check the above comment. Please go ahead and close the issue if it was resolved.Thanks!

[Deep-learning-practitioner]

Closing the issue but, I should say that in tf2 frozen graphs are depreciated but they are still posssible.

[zldrobit]

I found a way to convert TF Saved Model to frozen graph in TF2. Here is the code I excerpt from https://colab.research.google.com/drive/1It4V8R0E_3KLezMhXjWTBXcvEifP4aea?usp=sharing,

import tensorflow as tf

model_path = 'tf_model'

# load the saved_model using low-level API
m = tf.saved_model.load(model_path)

from tensorflow.python.framework.convert_to_constants import convert_variables_to_constants_v2

tfm = tf.function(lambda x: m(images=x))  # full model                                                  
tfm = tfm.get_concrete_function(tf.TensorSpec(m.signatures['serving_default'].inputs[0].shape.as_list(), m.signatures['serving_default'].inputs[0].dtype.name))   
frozen_func = convert_variables_to_constants_v2(tfm)                                                                                                                              
tf.io.write_graph(graph_or_graph_def=frozen_func.graph, logdir="./", name="yolov5n-onnx.pb", as_text=False)

.

[GuilhermeBassan]

I found a way to convert TF Saved Model to frozen graph in TF2. Here is the code I excerpt from https://colab.research.google.com/drive/1It4V8R0E_3KLezMhXjWTBXcvEifP4aea?usp=sharing,

import tensorflow as tf

model_path = 'tf_model'

# load the saved_model using low-level API
m = tf.saved_model.load(model_path)

from tensorflow.python.framework.convert_to_constants import convert_variables_to_constants_v2

tfm = tf.function(lambda x: m(images=x))  # full model                                                  
tfm = tfm.get_concrete_function(tf.TensorSpec(m.signatures['serving_default'].inputs[0].shape.as_list(), m.signatures['serving_default'].inputs[0].dtype.name))   
frozen_func = convert_variables_to_constants_v2(tfm)                                                                                                                              
tf.io.write_graph(graph_or_graph_def=frozen_func.graph, logdir="./", name="yolov5n-onnx.pb", as_text=False)

.

Hi @zldrobit, thanks for the contribution! Correct me if i'm wrong, but shouldn't tf.io.write_graph() be used to create the .pbtxt file instead the .pb frozen graph?

[zldrobit]

@GuilhermeBassan It could be used to save either a .pb or .pbtxt file. Setting as_text=True saves a .pbtxt file, otherwise it saves a .pb file. .pbtxt files are just the text counterpart of .pb files.

[GuilhermeBassan]

@GuilhermeBassan It could be used to save either a .pb or .pbtxt file. Setting as_text=True saves a .pbtxt file, otherwise it saves a .pb file. .pbtxt files are just the text counterpart of .pb files.

I see, it makes sense now. I used tf.io.write_graph() to generate a frozen graph from a model which I can use with tf.saved_model.load(), but was unable to use them for inference with OpenCV (I get error: (-215:Assertion failed) !empty() in function 'cv::dnn::dnn4_v20211220::Net::forward'). Am I missing something?

The code to generate the .pb I took from the one you posted here:

import argparse
import string
import tensorflow as tf
from tensorflow.python.framework.convert_to_constants import convert_variables_to_constants_v2
from tensorflow.python.tools import freeze_graph

def freeze(model, dir, name):
    m = tf.saved_model.load(model)

    tfm = tf.function(lambda x: m(x))  # full model                                                  
    tfm = tfm.get_concrete_function(
        tf.TensorSpec(
            m.signatures['serving_default'].inputs[0].shape.as_list(), 
            m.signatures['serving_default'].inputs[0].dtype.name
        )
    )   
    frozen_func = convert_variables_to_constants_v2(tfm)                                                                                                                              
    tf.io.write_graph(
        graph_or_graph_def=frozen_func.graph,
        logdir=dir, 
        name=name, 
        as_text=False
    )

def main():
    parser=argparse.ArgumentParser()
    parser.add_argument(
        "-m", "--model",
        help="Caminho para o modelo gerado",
        required=True
    )
    parser.add_argument(
        "-o", "--output",
        help="Arquivo de saída",
        required=True
    )
    parser.add_argument(
        "-n", "--name",
        help="Nome para o arquivo de saída (opcional)",
        default="frozen_model.pb"
    )
    args=parser.parse_args()

    freeze(args.model, args.output, args.name)

if __name__ == '__main__':
    main()

And the code to use it with opencv is as follows:

import cv2 as cv

path = "annotations/"

#cvNet = cv.dnn.readNetFromTensorflow(path+'frozen_model.pb', path+'graph.pbtxt')
cvNet = cv.dnn.readNetFromTensorflow(path+'frozen_model.pb')

img = cv.imread('scripts/example.jpg')
rows = img.shape[0]
cols = img.shape[1]
cvNet.setInput(cv.dnn.blobFromImage(img, size=(300, 300), swapRB=True, crop=False))
cvOut = cvNet.forward()

for detection in cvOut[0,0,:,:]:
    score = float(detection[2])
    if score > 0.3:
        left = detection[3] * cols
        top = detection[4] * rows
        right = detection[5] * cols
        bottom = detection[6] * rows
        cv.rectangle(img, (int(left), int(top)), (int(right), int(bottom)), (23, 230, 210), thickness=2)

cv.imshow('img', img)
cv.waitKey()

The complete error message is:

(venv) C:\Tensorflow\workspace>python scripts\test_graph.py
Traceback (most recent call last):
  File "scripts\test_graph.py", line 12, in <module>
    cvOut = cvNet.forward()
cv2.error: OpenCV(4.5.5) D:\a\opencv-python\opencv-python\opencv\modules\dnn\src\dnn.cpp:4481: error: (-215:Assertion failed) !empty() in function 'cv::dnn::dnn4_v20211220::Net::forward'

Any help will be greatly appreciated :D

[zldrobit]

@GuilhermeBassan I found an issue discussing the same problem, but it hasn't been updated for several months. There is also a tutorial about running .pb model with the OpenCV dnn module, and the conversion code in it is almost the same as I posted.

[GuilhermeBassan]

@GuilhermeBassan I found an issue discussing the same problem, but it hasn't been updated for several months. There is also a tutorial about running .pb model with the OpenCV dnn module, and the conversion code in it is almost the same as I posted.

Yeah I stumbled on those too on my researches, I've been trying to figure out this issue for a few days now.

Turns out I found this page and it states that tensorflow 2 does not export the pb models to be used with OpenCV. The tutorial is from 2 years ago so it might have changed by now, but I decided to go back to tf1 since I'm running out of time.

I am deeply thankful of your help!