Do you have caffe or tensorflow version?

[sharder996]

I'd like to try using this model on a mobile platform and have been trying to convert this model to tflite, but have have been running into problems. Has anyone else done this or know how to go about doing this?

These are the problems that I have run into:

• Converting the original keras model with TFLiteConverter fails as this repo is written in Tensorflow v1 and TFLiteConverter v1 doesn't support control flow operations which are used in the model. However, these operations are supported in v2.
• TFLiteConverter v2 doesn't have a method to convert from a keras model so the model has to be converted to another format. Converting to a saved model creates issues when specifying input and output tensors and I can never get a saved model that doesn't throw any errors.
• Converting this whole repo to Tensorflow v2 also throws errors from within the custom defined layers which I am not sure how to solve.

Code that I am using:

# Set the image size.
img_height = 512
img_width = 512

# Set the model's inference mode
model_mode = 'inference'

# Set the desired confidence threshold
conf_thresh = 0.01

# 1: Build the Keras model
K.clear_session() # Clear previous models from memory.
model = ssd_512(image_size=(img_height, img_width, 3),
                n_classes=1,
                mode=model_mode,
                l2_regularization=0.0005,
                scales=[0.07, 0.15, 0.3, 0.45, 0.6, 0.75, 0.9, 1.05], # PASCAL VOC
                aspect_ratios_per_layer=[[1.0, 2.0, 0.5],
                                         [1.0, 2.0, 0.5, 3.0, 1.0/3.0],
                                         [1.0, 2.0, 0.5, 3.0, 1.0/3.0],
                                         [1.0, 2.0, 0.5, 3.0, 1.0/3.0],
                                         [1.0, 2.0, 0.5, 3.0, 1.0/3.0],
                                         [1.0, 2.0, 0.5],
                                         [1.0, 2.0, 0.5]],
               two_boxes_for_ar1=True,
               steps=[8, 16, 32, 64, 128, 256, 512],
               offsets=[0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5],
               clip_boxes=False,
               variances=[0.1, 0.1, 0.2, 0.2],
               normalize_coords=True,
               subtract_mean=[123, 117, 104],
               swap_channels=[2, 1, 0],
               confidence_thresh=conf_thresh,
               iou_threshold=0.45,
               top_k=200,
               nms_max_output_size=400)

# 2: Load the trained weights into the model. Make sure the path correctly points to the model's .h5 file
weights_path = './data/ssd512-hollywood-trainval-bs_16-lr_1e-05-scale_pascal-epoch-187-py3.6.h5'
model.load_weights(weights_path, by_name=True)

# 3: Compile the model so that Keras won't complain the next time you load it.
adam = Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0)
ssd_loss = SSDLoss(neg_pos_ratio=3, alpha=1.0)
model.compile(optimizer=adam, loss=ssd_loss.compute_loss)

# Create an SSDLoss object in order to pass that to the model loader
ssd_loss = SSDLoss(neg_pos_ratio=3, n_neg_min=0, alpha=1.0)

# Clear previous models from memory.
K.clear_session() 

# Configure the decode detections layer based on the model mode
if model_mode == 'inference':
    decode_layer = DecodeDetections(img_height=img_height,
                                    img_width=img_width,
                                    confidence_thresh=conf_thresh,
                                    iou_threshold=0.45,
                                    top_k=200,
                                    nms_max_output_size=400)
if model_mode == 'inference_fast':
    decode_layer = DecodeDetectionsFast(img_height=img_height,
                                        img_width=img_width,
                                        confidence_thresh=conf_thresh,
                                        iou_threshold=0.45,
                                        top_k=200,
                                        nms_max_output_size=400)

# Finally load the model
model = load_model(weights_path, custom_objects={'AnchorBoxes': AnchorBoxes,
                                                 'L2Normalization': L2Normalization,
                                                 'DecodeDetections': decode_layer,
                                                 'compute_loss': ssd_loss.compute_loss})

builder = tf.saved_model.builder.SavedModelBuilder('./saved2')
with tf.gfile.GFile('./temp/model.pb', "rb") as f:
    graph_def = tf.GraphDef()
    graph_def.ParseFromString(f.read())

sigs = {}
with tf.Graph().as_default() as graph:
    # name="" is important to ensure we don't get spurious prefixing
    tf.import_graph_def(graph_def, name="")
    g = tf.get_default_graph()

    with tf.Session(graph=graph) as sess:

        op = sess.graph.get_operations()
        input_tensor = [m.values() for m in op][1][0]
        output_tensor = [m.values() for m in op][len(op)-1][0]

        tensor_info_input = tf.saved_model.utils.build_tensor_info(input_tensor)
        tensor_info_output = tf.saved_model.utils.build_tensor_info(output_tensor)

        sigs[signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY] = \
            tf.saved_model.signature_def_utils.build_signature_def(inputs={"input": tensor_info_input}, outputs={"input": tensor_info_output}, method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME)

        builder.add_meta_graph_and_variables(sess, [tag_constants.SERVING], signature_def_map=sigs)

builder.save()

print()
print('Successfully exited program')

The goal of the above code is save the model in a format where Tensorflow v2 can take over and the model to tflite. So, the final conversion would take place in a separate environment using Tensorflow v2.

import tensorflow as tf

saved_model_dir = './saved2/'
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.experimental_new_converter = True  # Add this line
tflite_model = converter.convert()

with open('model.tflite', 'wb') as f:
  f.write(tflite_model)

print('Successfully exited program')

While this script does finish without throwing any exceptions, the outputted tflite file size is on the order of bytes.

Any suggestions are welcome!

[sharder996]

Additionally, it would help a lot if the training script could be provided to aid in identifying the correct input and output nodes.

[mjmarin]

@sharder996 , we used the following code for training: https://github.com/pierluigiferrari/ssd_keras/

[asyarden]

@sharder996 any news on this? I'd be happy to help/get the .tflite weights file if you have it