I've trained a model, with great results, as follows:

model.load_weights('mask_rcnn_coco.h5', by_name=True, exclude=[
                       "mrcnn_class_logits", "mrcnn_bbox_fc",  "mrcnn_bbox", "mrcnn_mask"])
model.train(train_set, test_set, learning_rate=config.LEARNING_RATE, epochs=num_epochs, augmentation=augmentation, layers='heads')

It trains the last layer (heads) starting from mask_rcnn_coco.h5.

Now I want to export my model to a .pb-file. The goal is to create a Tensorflow Lite file to be used in a mobile application.

I've tried using 'keras_to_tensorflow.py'

https://github.com/amir-abdi/keras_to_tensorflow/blob/master/keras_to_tensorflow.py

But get the error

ValueError: Unknown layer: BatchNorm

I've tried freeze_model

https://github.com/matterport/Mask_RCNN/issues/1538#issuecomment-501932017

But get this error

Attempting to use uninitialized value bn4m_branch2a/moving_mean

I've set save_weights_only to False

save_weights_only=False

But got this error: TypeError: can't pickle _thread.RLock objects

I've tried this suggestion:

https://github.com/matterport/Mask_RCNN/issues/1538#issuecomment-501932017

But got this error: Attempting to use uninitialized value bn4m_branch2a/moving_mean

Based on this 152-comments-strong thread https://github.com/matterport/Mask_RCNN/issues/218 it seems like a lot of people have the same or similar issues without any clear answer.

Are there any other paths I could explore in order to save my model as a .pb-file?
How are other people using this model in production?
Are there other similar models out there where exporting is possible?

Hi @davidvirdeforshh, I made a script to save the whole model in .h5 format that .pb, try it and let me know. I'm also working to bring the model on the Intel Movidus or Google Coral devboard

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import os
import sys
import warnings

import keras.backend as K
import tensorflow as tf

warnings.filterwarnings('ignore', category=FutureWarning)
# suppress warning and error message tf
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"

# Root directory of the project
ROOT_DIR = os.getcwd()
# Import Mask RCNN
sys.path.append(ROOT_DIR)  # To find local version of the library
from mrcnn import model as modellib
from mrcnn import utils
import landingzone

K.clear_session()
K.set_learning_phase(0)

##############################################################################
# Load model
##############################################################################

# Model Directory
MODEL_DIR = os.path.join(ROOT_DIR, "logs")
DEFAULT_WEIGHTS = os.path.join(ROOT_DIR, "mask_rcnn_coco.h5")
# Download COCO trained weights from Releases if needed
if not os.path.exists(DEFAULT_WEIGHTS):
    utils.download_trained_weights(DEFAULT_WEIGHTS)

##############################################################################
# Load configuration
##############################################################################

# Load Configuration Model
config = landingzone.LandingZoneConfig()

# Override the training configurations with a few changes for inference.
class InferenceConfig(config.__class__):
    # Run detection on one image at a time
    GPU_COUNT = 1
    IMAGES_PER_GPU = 1

# Device to load the neural network on.
# Useful if you're training a model on the same
# machine, in which case use CPU and leave the
# GPU for training.
DEVICE = "/cpu:0"

# Inspect the model in training or inference modes
# values: 'inference' or 'training'
TEST_MODE = "inference"

##############################################################################
# Save entire model function
##############################################################################

def h5_to_pb(h5_model, output_dir, model_name, out_prefix="output_"):
    out_nodes = []
    for i in range(len(h5_model.outputs)):
        out_nodes.append(out_prefix + str(i + 1))
        tf.identity(h5_model.output[i], out_prefix + str(i + 1))
    sess = K.get_session()
    init_graph = sess.graph.as_graph_def()
    main_graph = tf.compat.v1.graph_util.convert_variables_to_constants(sess, init_graph, out_nodes)
    with tf.gfile.GFile(os.path.join(output_dir, model_name), "wb") as filemodel:
        filemodel.write(main_graph.SerializeToString())
    print(f"pb model: {os.path.join(output_dir, model_name)}")

if __name__ == "__main__":
    config = InferenceConfig()
    config.display()
    # Create model in inference mode
    with tf.device(DEVICE):
        model = modellib.MaskRCNN(
            mode=TEST_MODE, model_dir=MODEL_DIR, config=config)

    # Set path to model weights
    weights_path = model.find_last()
    # Load weights
    print("Loading weights ", weights_path)
    model.load_weights(weights_path, by_name=True)
    model.keras_model.summary()

    # make folder for full model
    model_dir = os.path.join(ROOT_DIR, "Model")
    if not os.path.exists(model_dir):
        os.makedirs(model_dir)

    # save h5 full model
    name_model = os.path.join(model_dir, "mask_rcnn_landing.h5")
    if not os.path.exists(name_model):
        model.keras_model.save(name_model)
        print(f"save model: {name_model}")

    # export pb model
    pb_name_model = "mask_rcnn_landing.pb"
    h5_to_pb(model.keras_model, output_dir=model_dir, model_name=pb_name_model)
    K.clear_session()
    sys.exit()

Hi @davidvirdeforshh, I made a script to save the whole model in .h5 format that .pb, try it and let me know. I'm also working to bring the model on the Intel Movidus or Google Coral devboard

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import os
import sys
import warnings

import keras.backend as K
import tensorflow as tf

warnings.filterwarnings('ignore', category=FutureWarning)
# suppress warning and error message tf
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"

# Root directory of the project
ROOT_DIR = os.getcwd()
# Import Mask RCNN
sys.path.append(ROOT_DIR)  # To find local version of the library
from mrcnn import model as modellib
from mrcnn import utils
import landingzone

K.clear_session()
K.set_learning_phase(0)

##############################################################################
# Load model
##############################################################################

# Model Directory
MODEL_DIR = os.path.join(ROOT_DIR, "logs")
DEFAULT_WEIGHTS = os.path.join(ROOT_DIR, "mask_rcnn_coco.h5")
# Download COCO trained weights from Releases if needed
if not os.path.exists(DEFAULT_WEIGHTS):
    utils.download_trained_weights(DEFAULT_WEIGHTS)

##############################################################################
# Load configuration
##############################################################################

# Load Configuration Model
config = landingzone.LandingZoneConfig()

# Override the training configurations with a few changes for inference.
class InferenceConfig(config.__class__):
    # Run detection on one image at a time
    GPU_COUNT = 1
    IMAGES_PER_GPU = 1

# Device to load the neural network on.
# Useful if you're training a model on the same
# machine, in which case use CPU and leave the
# GPU for training.
DEVICE = "/cpu:0"

# Inspect the model in training or inference modes
# values: 'inference' or 'training'
TEST_MODE = "inference"

##############################################################################
# Save entire model function
##############################################################################

def h5_to_pb(h5_model, output_dir, model_name, out_prefix="output_"):
    out_nodes = []
    for i in range(len(h5_model.outputs)):
        out_nodes.append(out_prefix + str(i + 1))
        tf.identity(h5_model.output[i], out_prefix + str(i + 1))
    sess = K.get_session()
    init_graph = sess.graph.as_graph_def()
    main_graph = tf.compat.v1.graph_util.convert_variables_to_constants(sess, init_graph, out_nodes)
    with tf.gfile.GFile(os.path.join(output_dir, model_name), "wb") as filemodel:
        filemodel.write(main_graph.SerializeToString())
    print(f"pb model: {os.path.join(output_dir, model_name)}")

if __name__ == "__main__":
    config = InferenceConfig()
    config.display()
    # Create model in inference mode
    with tf.device(DEVICE):
        model = modellib.MaskRCNN(
            mode=TEST_MODE, model_dir=MODEL_DIR, config=config)

    # Set path to model weights
    weights_path = model.find_last()
    # Load weights
    print("Loading weights ", weights_path)
    model.load_weights(weights_path, by_name=True)
    model.keras_model.summary()

    # make folder for full model
    model_dir = os.path.join(ROOT_DIR, "Model")
    if not os.path.exists(model_dir):
        os.makedirs(model_dir)

    # save h5 full model
    name_model = os.path.join(model_dir, "mask_rcnn_landing.h5")
    if not os.path.exists(name_model):
        model.keras_model.save(name_model)
        print(f"save model: {name_model}")

    # export pb model
    pb_name_model = "mask_rcnn_landing.pb"
    h5_to_pb(model.keras_model, output_dir=model_dir, model_name=pb_name_model)
    K.clear_session()
    sys.exit()

Hi frank. I did all that you said and refactor my code to tensorflow 2.01 , but when came to below line tf.identity(h5_model.output[i], out_prefix + str(i + 1)) tensrflow doesnt add to session graph and throw exception when call convert_variables_to_constants Have you any idea about it exception : AssertionError: output_1 is not in graph Here my code

def h5_to_pb(h5_model, output_dir, model_name, out_prefix="output_"):
    out_nodes = []
    for i in range(len(h5_model.outputs)):
        out_nodes.append(out_prefix + str(i + 1))
        tf.identity(h5_model.output[i], out_prefix + str(i + 1))
    sess = tf.compat.v1.keras.backend.get_session()
    init_graph = sess.graph.as_graph_def()
    main_graph = tf.compat.v1.graph_util.convert_variables_to_constants(sess, init_graph, out_nodes)
    with tf.io.gfile.GFile(os.path.join(output_dir, model_name), "wb") as filemodel:
        filemodel.write(init_graph.SerializeToString())
    print(f"pb model: {os.path.join(output_dir, model_name)}")

No idea, I have used with tensorflow 1.14, but i suggest use netron to explore h5 file and fix the problem.

!/usr/bin/env python3

-- coding: utf-8 --

import os import sys import warnings

import keras.backend as K import tensorflow as tf

warnings.filterwarnings('ignore', category=FutureWarning)

suppress warning and error message tf

os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"

Root directory of the project

ROOT_DIR = os.getcwd()

Import Mask RCNN

sys.path.append(ROOT_DIR) # To find local version of the library from mrcnn import model as modellib from mrcnn import utils import landingzone

K.clear_session() K.set_learning_phase(0)

##############################################################################

Load model

##############################################################################

Model Directory

MODEL_DIR = os.path.join(ROOT_DIR, "logs") DEFAULT_WEIGHTS = os.path.join(ROOT_DIR, "mask_rcnn_coco.h5")

Download COCO trained weights from Releases if needed

if not os.path.exists(DEFAULT_WEIGHTS): utils.download_trained_weights(DEFAULT_WEIGHTS)

##############################################################################

Load configuration

##############################################################################

Load Configuration Model

config = landingzone.LandingZoneConfig()

Override the training configurations with a few changes for inference.

class InferenceConfig(config.class):

Run detection on one image at a time
GPU_COUNT = 1
IMAGES_PER_GPU = 1
Device to load the neural network on.

Useful if you're training a model on the same

machine, in which case use CPU and leave the

GPU for training.

DEVICE = "/cpu:0"

Inspect the model in training or inference modes

values: 'inference' or 'training'

TEST_MODE = "inference"

##############################################################################

Save entire model function

##############################################################################

def h5_to_pb(h5_model, output_dir, model_name, outprefix="output"): out_nodes = [] for i in range(len(h5_model.outputs)): out_nodes.append(out_prefix + str(i + 1)) tf.identity(h5_model.output[i], out_prefix + str(i + 1)) sess = K.get_session() init_graph = sess.graph.as_graph_def() main_graph = tf.compat.v1.graph_util.convert_variables_to_constants(sess, init_graph, out_nodes) with tf.gfile.GFile(os.path.join(output_dir, model_name), "wb") as filemodel: filemodel.write(main_graph.SerializeToString()) print(f"pb model: {os.path.join(output_dir, model_name)}")

if name == "main": config = InferenceConfig() config.display()

Create model in inference mode
with tf.device(DEVICE):
    model = modellib.MaskRCNN(
        mode=TEST_MODE, model_dir=MODEL_DIR, config=config)

# Set path to model weights
weights_path = model.find_last()
# Load weights
print("Loading weights ", weights_path)
model.load_weights(weights_path, by_name=True)
model.keras_model.summary()

# make folder for full model
model_dir = os.path.join(ROOT_DIR, "Model")
if not os.path.exists(model_dir):
    os.makedirs(model_dir)

# save h5 full model
name_model = os.path.join(model_dir, "mask_rcnn_landing.h5")
if not os.path.exists(name_model):
    model.keras_model.save(name_model)
    print(f"save model: {name_model}")

# export pb model
pb_name_model = "mask_rcnn_landing.pb"
h5_to_pb(model.keras_model, output_dir=model_dir, model_name=pb_name_model)
K.clear_session()
sys.exit()
No idea, I have used with tensorflow 1.14, but i suggest use [netron](https://github.com/lutzroeder/netron) to explore h5 file and fix the problem.

Hi, @frank1789 I ran your script but had the error of "No module named 'landingzone'. And the pip install didn't work, either. Where can I download the landingzone package?

Hey @jisooyu and sorry @frank1789 I'm answering for you. Landing zone is actually your own file, like balloon for example. He named it landing zone cause this is what he wants to detect with MaskRCNN. Bring in your own config.

Hi, @jisooyu exactly what i wanted reply.

Hi, @frank1789 and @suchiz, thank you for your help. Then it should be the h5 file, right?

@frank1789 and @suchiz, My problem is that my own h5 file is located in the logs directory under the Mask_RCNN. But the h5 is locked and can't be accessed it no matter what I tried. Could you please show me how to save the h5 file in the designated directory? And of course, it should be accessible. Thank you for your help.

i will try explain better:

file named "landingzone.py" is a file configuration for training or run inference -> replace with yours (*.py)
file "mask_rcnn_coco.h5" is default file content standard model and weight.
file "mask_rcnn_landing.h5" is your file resulting from training process. -> (in your case "maskrcnn*.h5")

then executing this script you can achieve a "mask_rcnn_landing.h5.pb" file format. ->(in your case "maskrcnn*.pb")

@frank1789 and @suchiz, My problem is that my own h5 file is located in the logs directory under the Mask_RCNN. But the h5 is locked and can't be accessed it no matter what I tried. Could you please show me how to save the h5 file in the designated directory? And of course, it should be accessible. Thank you for your help.

If h5 file result blocked, maybe you use original script. I have introduced modification to run on HPC during training, so you can replace the script inside library. Use this file to update mrcnn folder and let me know. mrcnn.zip

Dear @frank1789 , I think I do now understand. I will try what you have explained and get back to you. Many thanks for your help.

convert-to pb file

#!encoding=utf-8
'''
#-----------------
Authors:邹九
Time:2019-11-21
#-----------------
'''
"""
Copyright (c) 2019, by the Authors: Amir H. Abdi
This script is freely available under the MIT Public License.
Please see the License file in the root for details.

The following code snippet will convert the keras model files
to the freezed .pb tensorflow weight file. The resultant TensorFlow model
holds both the model architecture and its associated weights.
"""

import tensorflow as tf
from tensorflow.python.framework import graph_util
from tensorflow.python.framework import graph_io
from pathlib import Path
from absl import app
from absl import flags
from absl import logging
from mrcnn import model as modellib
from mrcnn.config import Config
import keras
import os
from keras import backend as K
from keras.models import model_from_json, model_from_yaml
from keras.utils.vis_utils import plot_model

COCO_MODEL_PATH = r'../logs/shapes20191113T1540_mask_rcnn_shapes_0199.h5'

K.set_learning_phase(0)
FLAGS = flags.FLAGS

flags.DEFINE_string('input_model', default=r'', help='Path to the input model.')
flags.DEFINE_string('input_model_json', None, 'Path to the input model '
                                              'architecture in json format.')
flags.DEFINE_string('input_model_yaml', None, 'Path to the input model architecture in yaml format.')
flags.DEFINE_string('output_model', default=r'./shapes20191113T1540_mask_rcnn_shapes_0199.pb', help='Path where the converted model will be stored.')
flags.DEFINE_boolean('save_graph_def', False,
                     'Whether to save the graphdef.pbtxt file which contains '
                     'the graph definition in ASCII format.')
flags.DEFINE_string('output_nodes_prefix', None,
                    'If set, the output nodes will be renamed to '
                    '`output_nodes_prefix`+i, where `i` will numerate the '
                    'number of of output nodes of the network.')
flags.DEFINE_boolean('quantize', False,
                     'If set, the resultant TensorFlow graph weights will be '
                     'converted from float into eight-bit equivalents. See '
                     'documentation here: '
                     'https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/graph_transforms')
flags.DEFINE_boolean('channels_first', False,
                     'Whether channels are the first dimension of a tensor. '
                     'The default is TensorFlow behaviour where channels are '
                     'the last dimension.')
flags.DEFINE_boolean('output_meta_ckpt', False,
                     'If set to True, exports the model as .meta, .index, and '
                     '.data files, with a checkpoint file. These can be later '
                     'loaded in TensorFlow to continue training.')

flags.mark_flag_as_required('input_model')
flags.mark_flag_as_required('output_model')

def load_model(input_model_path, input_json_path=None, input_yaml_path=None):
    if not Path(input_model_path).exists():
        raise FileNotFoundError(
            'Model file `{}` does not exist.'.format(input_model_path))
    try:
        model = keras.models.load_model(input_model_path)
        return model
    except FileNotFoundError as err:
        logging.error('Input mode file (%s) does not exist.', FLAGS.input_model)
        raise err
    except ValueError as wrong_file_err:
        if input_json_path:
            if not Path(input_json_path).exists():
                raise FileNotFoundError(
                    'Model description json file `{}` does not exist.'.format(
                        input_json_path))
            try:
                model = model_from_json(open(str(input_json_path)).read())
                model.load_weights(input_model_path)
                return model
            except Exception as err:
                logging.error("Couldn't load model from json.")
                raise err
        elif input_yaml_path:
            if not Path(input_yaml_path).exists():
                raise FileNotFoundError(
                    'Model description yaml file `{}` does not exist.'.format(
                        input_yaml_path))
            try:
                model = model_from_yaml(open(str(input_yaml_path)).read())
                model.load_weights(input_model_path)
                return model
            except Exception as err:
                logging.error("Couldn't load model from yaml.")
                raise err
        else:
            logging.error(
                'Input file specified only holds the weights, and not '
                'the model definition. Save the model using '
                'model.save(filename.h5) which will contain the network '
                'architecture as well as its weights. '
                'If the model is saved using the '
                'model.save_weights(filename) function, either '
                'input_model_json or input_model_yaml flags should be set to '
                'to import the network architecture prior to loading the '
                'weights. \n'
                'Check the keras documentation for more details '
                '(https://keras.io/getting-started/faq/)')
            raise wrong_file_err

class ShapesConfig(Config):
    """Configuration for training on the toy shapes dataset.
    Derives from the base Config class and overrides values specific
    to the toy shapes dataset.
    """
    # Give the configuration a recognizable name
    NAME = "shapes"

    # Number of classes (including background)
    NUM_CLASSES = 1 + 14  # background + 15 object
    # Choose the number of GPU devices
    # os.environ['CUDA_VISIBLE_DEVICES'] = '0'

    # Use small images for faster training. Set the limits of the small side
    # the large side, and that determines the image shape.
    IMAGE_RESIZE_MODE = "square"
    IMAGE_MAX_DIM = 896

    RPN_ANCHOR_SCALES = (8 * 6, 16 * 6, 32 * 6, 64 * 6, 128 * 6)  # anchor side in pixels
    # RPN_ANCHOR_SCALES = (8*5, 16*5, 32*5, 64*5, 128*5)  # anchor side in pixels

    # Reduce training ROIs per image because the images are small and have
    # few objects. Aim to allow ROI sampling to pick 33% positive ROIs.
    TRAIN_ROIS_PER_IMAGE = 100

    # Use a small epoch since the data is simple
    # STEPS_PER_EPOCH = 1000
    STEPS_PER_EPOCH = 1000

    # use small validation steps since the epoch is small
    VALIDATION_STEPS = 25

def main(args):
    # If output_model path is relative and in cwd, make it absolute from root
    output_model = FLAGS.output_model
    if str(Path(output_model).parent) == '.':
        output_model = str((Path.cwd() / output_model))

    output_fld = Path(output_model).parent
    output_model_name = Path(output_model).name
    output_model_stem = Path(output_model).stem
    output_model_pbtxt_name = output_model_stem + '.pbtxt'

    # Create output directory if it does not exist
    Path(output_model).parent.mkdir(parents=True, exist_ok=True)

    if FLAGS.channels_first:
        K.set_image_data_format('channels_first')
    else:
        K.set_image_data_format('channels_last')

    # model = load_model(FLAGS.input_model, FLAGS.input_model_json, FLAGS.input_model_yaml)
    ##--------------------------------------------------------------------------------------#
    config = ShapesConfig()
    config.display()
    MODEL_DIR = r'E:\Desktop\Projects\Mask_RCNN-master\logs'
    model = modellib.MaskRCNN(mode="inference", config=config,\
                              model_dir=MODEL_DIR)
    model.load_weights(COCO_MODEL_PATH, by_name=True)#exclude=["mrcnn_class_logits", "mrcnn_bbox_fc",\
                                # "mrcnn_bbox", "mrcnn_mask"])
    # print(model.summary())
    # plot_model(model, to_file='model1.png', show_shapes=True)
    # model_json = model.to_json()
    # with open(r'./modle.json', 'w') as file:
    #     file.write(model_json)

    print('loaded model and saved json file')
    ##--------------------------------------------------------------------------------------#
    # TODO(amirabdi): Support networks with multiple inputs
    # orig_output_node_names = [node.op.name for node in model.outputs]
    orig_output_node_names = ['mrcnn_detection/Reshape_1', 'mrcnn_class/Softmax', 'mrcnn_bbox/Reshape',\
                              'mrcnn_mask/Sigmoid', 'ROI/packed_2', 'rpn_class/concat', 'rpn_bbox/concat']

    if FLAGS.output_nodes_prefix:
        num_output = len(orig_output_node_names)
        pred = [None] * num_output
        converted_output_node_names = [None] * num_output

        # Create dummy tf nodes to rename output
        for i in range(num_output):
            converted_output_node_names[i] = '{}{}'.format(
                FLAGS.output_nodes_prefix, i)
            pred[i] = tf.identity(model.outputs[i],
                                  name=converted_output_node_names[i])
    else:
        converted_output_node_names = orig_output_node_names
    logging.info('Converted output node names are: %s',
                 str(converted_output_node_names))

    sess = K.get_session()
    if FLAGS.output_meta_ckpt:
        saver = tf.train.Saver()
        saver.save(sess, str(output_fld / output_model_stem))

    if FLAGS.save_graph_def:
        tf.train.write_graph(sess.graph.as_graph_def(), str(output_fld),
                             output_model_pbtxt_name, as_text=True)
        logging.info('Saved the graph definition in ascii format at %s',
                     str(Path(output_fld) / output_model_pbtxt_name))

    if FLAGS.quantize:
        from tensorflow.tools.graph_transforms import TransformGraph
        transforms = ["quantize_weights", "quantize_nodes"]
        transformed_graph_def = TransformGraph(sess.graph.as_graph_def(), [],
                                               converted_output_node_names,
                                               transforms)
        constant_graph = graph_util.convert_variables_to_constants(
            sess,
            transformed_graph_def,
            converted_output_node_names)
    else:
        constant_graph = graph_util.convert_variables_to_constants(
            sess,
            sess.graph.as_graph_def(),
            converted_output_node_names)

    graph_io.write_graph(constant_graph, str(output_fld), output_model_name,
                         as_text=False)
    logging.info('Saved the freezed graph at %s',
                 str(Path(output_fld) / output_model_name))

if __name__ == "__main__":
    app.run(main)

load pb model

def load_detection_model(model):
    config = tf.ConfigProto()
    config.gpu_options.allow_growth = True
    detection_graph = tf.Graph()
    with detection_graph.as_default():
        od_graph_def = tf.GraphDef()
        with tf.gfile.GFile(model, 'rb') as fid:
            serialized_graph = fid.read()
            od_graph_def.ParseFromString(serialized_graph)
            tf.import_graph_def(od_graph_def, name='')
        input_image = tf.get_default_graph().get_tensor_by_name('input_image:0')
        input_image_meta = tf.get_default_graph().get_tensor_by_name('input_image_meta:0')
        input_anchors = tf.get_default_graph().get_tensor_by_name('input_anchors:0')
        detections = tf.get_default_graph().get_tensor_by_name('mrcnn_detection/Reshape_1:0')
        mrcnn_mask = tf.get_default_graph().get_tensor_by_name('mrcnn_mask/Sigmoid:0')
    sessd=tf.Session(config=config,graph=detection_graph)
    print('Loaded detection model from file "%s"' % model)
    return sessd, input_image, input_image_meta, input_anchors, detections, mrcnn_mask

sessd, input_image, input_image_meta, input_anchors, detections, mrcnn_mask = load_detection_model(model_path)
results = model.detect_pb([image], sessd, input_image, input_image_meta, input_anchors, detections, mrcnn_mask,verbose=1)

use model, add to mrcnn/model.py

    def detect_pb(self, images, sessd, input_image, input_image_meta, input_anchors, detections, mrcnn_mask, verbose=1):
        """Runs the detection pipeline.

        images: List of images, potentially of different sizes.

        Returns a list of dicts, one dict per image. The dict contains:
        rois: [N, (y1, x1, y2, x2)] detection bounding boxes
        class_ids: [N] int class IDs
        scores: [N] float probability scores for the class IDs
        masks: [H, W, N] instance binary masks
        """
        assert self.mode == "inference", "Create model in inference mode."
        assert len(
            images) == self.config.BATCH_SIZE, "len(images) must be equal to BATCH_SIZE"

        # if verbose:
        #     log("Processing {} images".format(len(images)))
        #     for image in images:
        #         log("image", image)

        # Mold inputs to format expected by the neural network
        molded_images, image_metas, windows = self.mold_inputs(images)

        # Validate image sizes
        # All images in a batch MUST be of the same size
        image_shape = molded_images[0].shape
        # print(image_shape, molded_images.shape)
        for g in molded_images[1:]:
            assert g.shape == image_shape,\
                "After resizing, all images must have the same size. Check IMAGE_RESIZE_MODE and image sizes."

        # Anchors
        anchors = self.get_anchors(image_shape)
        # Duplicate across the batch dimension because Keras requires it
        # TODO: can this be optimized to avoid duplicating the anchors?
        anchors = np.broadcast_to(anchors, (self.config.BATCH_SIZE,) + anchors.shape)

        # if verbose:
        #     log("molded_images", molded_images)
        #     log("image_metas", image_metas)
        #     log("anchors", anchors)
        # Run object detection
        # detections, _, _, mrcnn_mask, _, _, _ =\
        #     self.keras_model.predict([molded_images, image_metas, anchors], verbose=0)
        detectionsed, mrcnn_masked = sessd.run([detections, mrcnn_mask], feed_dict = {input_image: molded_images, \
                                                               input_image_meta: image_metas, \
                                                               input_anchors: anchors})
        mrcnn_masked = np.expand_dims(mrcnn_masked, 0)
        detections = np.array(detectionsed)
        mrcnn_mask = np.array(mrcnn_masked)
        # Process detections
        results = []
        for i, image in enumerate(images):
            xi = detections[i]
            yi = mrcnn_mask[i]
            moldedi = molded_images[i]
            windowsi = windows[i]
            final_rois, final_class_ids, final_scores, final_masks =\
                self.unmold_detections(detections[i], mrcnn_mask[i],
                                       image.shape, molded_images[i].shape,
                                       windows[i])
            results.append({
                "rois": final_rois,
                "class_ids": final_class_ids,
                "scores": final_scores,
                "masks": final_masks,
            })
        return results

Hello, so I tested the script that @frank1789 provided and I get this error:

Traceback (most recent call last): File "mask_rcnn_h5_to_pb.py", line 88, in weights_path = model.find_last() File "/usr/local/lib/python3.6/dist-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 2073, in find_last StopIteration

I'm not too sure how to solve this problem, could I get some help please?

Hello, so I tested the script that @frank1789 provided and I get this error:

Traceback (most recent call last): File "mask_rcnn_h5_to_pb.py", line 88, in weights_path = model.find_last() File "/usr/local/lib/python3.6/dist-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 2073, in find_last StopIteration

I'm not too sure how to solve this problem, could I get some help please?

Well, you have to modify file in mrcnn/model.py in particular method __find_last()__ as reported below:

    def find_last(self):
        """Finds the last checkpoint file of the last trained model in the
        model directory.
        Returns:
            The path of the last checkpoint file
        """
        # Get directory names. Each directory corresponds to a model
        dir_names = next(os.walk(self.model_dir))[1]
        key = self.config.NAME.lower()
        dir_names = filter(lambda f: f.startswith(key), dir_names)
        dir_names = sorted(dir_names)
        if not dir_names:
            import errno
            raise FileNotFoundError(
                errno.ENOENT,
                "Could not find model directory under {}".format(self.model_dir))
        # Pick last directory
        dir_name = os.path.join(self.model_dir, dir_names[-1])
        # Find the last checkpoint
        checkpoints = next(os.walk(dir_name))[2]
        checkpoints = filter(lambda f: f.startswith("mask_rcnn"), checkpoints)
        checkpoints = sorted(checkpoints)
        if not checkpoints:
            import errno
            raise FileNotFoundError(
                errno.ENOENT, "Could not find weight files in {}".format(dir_name))
        checkpoint = os.path.join(dir_name, checkpoints[-1])
        return checkpoint

Hope this can fix your issue.

shiheyingzhe,

I used your scripts to convert mask_rcnn_coco.h5 to pb. And that filename is pretty much i changed in the script. I then added a evaluate_coco_pb in samples/coco/coco.py (see below) which use the two functions you prvided model.detect_pb and load_detection_model('./samples/coco/mask_rcnn_coco.pb'). But some how at run time, I got an error that shape size does not match. Is there something i need to modify for coco when converting pb file from h5? Please advise, thank you very much.

Traceback (most recent call last): File "./samples/coco/coco.py", line 615, in evaluate_coco_pb(model, dataset_val, coco, "bbox", limit=int(args.limit)) File "./samples/coco/coco.py", line 447, in evaluate_coco_pb r = model.detect_pb([image], sessd, input_image, input_image_meta, input_anchors, detections, mrcnn_mask,verbose=1)[0] File "/home/vl933734/venv1152/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 2768, in detect_pb File "/home/vl933734/venv1152/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 956, in run run_metadata_ptr) File "/home/vl933734/venv1152/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1156, in _run (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape()))) ValueError: Cannot feed value of shape (1, 93) for Tensor 'input_image_meta:0', which has shape '(?, 27)'

def evaluate_coco_pb(model, dataset, coco, eval_type="bbox", limit=0, image_ids=None): """Runs official COCO evaluation. dataset: A Dataset object with valiadtion data eval_type: "bbox" or "segm" for bounding box or segmentation evaluation limit: if not 0, it's the number of images to use for evaluation """

# Pick COCO images from the dataset
image_ids = image_ids or dataset.image_ids

# Limit to a subset
if limit:
    image_ids = image_ids[:limit]

# Get corresponding COCO image IDs.
coco_image_ids = [dataset.image_info[id]["id"] for id in image_ids]

t_prediction = 0
t_start = time.time()

results = []
for i, image_id in enumerate(image_ids):
    # Load image
    image = dataset.load_image(image_id)

    # Run detection
    t = time.time()
    #r = model.detect([image], verbose=0)[0]
    sessd, input_image, input_image_meta, input_anchors, detections, mrcnn_mask = load_detection_model('./samples/coco/mask_rcnn_coco.pb')
    r = model.detect_pb([image], sessd, input_image, input_image_meta, input_anchors, detections, mrcnn_mask,verbose=1)[0] 
    t_prediction += (time.time() - t)

    # Convert results to COCO format
    # Cast masks to uint8 because COCO tools errors out on bool
    image_results = build_coco_results(dataset, coco_image_ids[i:i + 1],
                                       r["rois"], r["class_ids"],
                                       r["scores"],
                                       r["masks"].astype(np.uint8))
    results.extend(image_results)

# Load results. This modifies results with additional attributes.
coco_results = coco.loadRes(results)

# Evaluate
cocoEval = COCOeval(coco, coco_results, eval_type)
cocoEval.params.imgIds = coco_image_ids
cocoEval.evaluate()
cocoEval.accumulate()
cocoEval.summarize()

print("Prediction time: {}. Average {}/image".format(
    t_prediction, t_prediction / len(image_ids)))
print("Total time: ", time.time() - t_start)

hello, @frank1789 I used materport's mask RCNN training to get the H5 weight, By the way, when saving H5 files,the program seems to use save weights Only = true. In this case, can I use the code you provided to convert H5 to Pb? Thank you very much!

Callbacks

    callbacks = [
        keras.callbacks.TensorBoard(log_dir=self.log_dir,
                                    histogram_freq=0, write_graph=True, write_images=False),
        keras.callbacks.ModelCheckpoint(self.checkpoint_path,
                                        verbose=0, save_weights_only=True),
    ]

I'm quite confident it is possible. Furthermore, you can compare the code with my custom code, a zip file shared some that I posted before.(https://github.com/matterport/Mask_RCNN/issues/1701#issuecomment-621064620)

@frank1789 OK, I will try to convert my H5 file to Pb according to your prompt. Thank you again for your reply

@frank1789 Hello, what is the version of keras and Tensorflow? And can the converted PB file be used for loading OpencV DNN module? Could you please tell me

matterport / Mask_RCNN

Can not save full model as .pb-file. #1701

I've tried using 'keras_to_tensorflow.py'

I've tried freeze_model

I've set save_weights_only to False

I've tried this suggestion:

!/usr/bin/env python3

-- coding: utf-8 --

suppress warning and error message tf

Root directory of the project

Import Mask RCNN

Load model

Model Directory

Download COCO trained weights from Releases if needed

Load configuration

Load Configuration Model

Override the training configurations with a few changes for inference.

Run detection on one image at a time

Device to load the neural network on.

Useful if you're training a model on the same

machine, in which case use CPU and leave the

GPU for training.

Inspect the model in training or inference modes

values: 'inference' or 'training'

Save entire model function

Create model in inference mode

Callbacks