how to deploy MaskRCNN to TF_Serving

[aashish-0393]

Hi

I was trying to save the full model with weights and deploy it to TF-Servings.

Can someone guide mw in how to save full model with weights and deploy it on Servings

[bendangnuksung]

@aashish-0393 have you found a way?

[adebiasio-estilos]

I have the same problem.

[adebiasio-estilos]

got the .pb model with the following:

sess = K.get_session() tf.saved_model.simple_save( sess, "your_model_pb_dir", inputs={t.name: t for t in model.keras_model.inputs}, outputs={t.name: t for t in model.keras_model.outputs})

[adebiasio-estilos]

The problem is that reloading the model i have a unexpected result:

TRADITIONAL BEHAVIOUR: {'rois': array([[ 391, 565, 554, 646], [ 643, 616, 792, 688], [ 390, 648, 551, 731], [ 386, 482, 552, 564], [ 131, 619, 306, 711], [ 646, 535, 793, 611], [ 140, 522, 310, 609], [ 639, 693, 795, 769], [ 417, 730, 550, 788], [ 376, 788, 545, 863], [1296, 621, 1364, 705], [ 859, 301, 911, 430], [ 912, 304, 1049, 404], [1335, 712, 1375, 787], [1130, 707, 1311, 791], [ 655, 830, 798, 886], [1224, 622, 1293, 700], [ 650, 771, 793, 834], [1197, 456, 1321, 543], [1288, 791, 1351, 881], [ 144, 249, 306, 336], [1129, 789, 1296, 889], [1121, 710, 1157, 791], [1169, 371, 1203, 452], [1163, 537, 1292, 602], [1112, 318, 1168, 435], [1185, 372, 1297, 455], [1281, 385, 1356, 458], [ 180, 812, 300, 903], [1124, 800, 1153, 885], [1171, 282, 1288, 375], [1248, 710, 1302, 754], [1270, 293, 1367, 384], [ 701, 211, 754, 268], [ 632, 969, 747, 1017], [1126, 619, 1214, 701], [ 622, 765, 635, 831], [ 626, 625, 641, 692], [ 29, 1015, 113, 1106], [ 584, 1124, 617, 1174], [ 641, 1028, 751, 1074], [1166, 289, 1197, 354], [1289, 461, 1362, 535], [ 578, 199, 628, 241], [1258, 540, 1365, 615], [ 24, 1077, 113, 1117], [ 119, 718, 179, 760], [ 501, 1155, 577, 1180], [ 135, 1001, 237, 1080], [1183, 462, 1219, 540], [ 444, 972, 549, 1109], [ 430, 207, 471, 248], [ 173, 1076, 245, 1102], [1143, 790, 1188, 882], [ 237, 1016, 438, 1090], [ 191, 418, 305, 491], [ 19, 963, 96, 1005], [ 393, 182, 474, 245], [ 583, 1036, 618, 1092], [ 901, 308, 924, 408], [1156, 713, 1182, 792], [1121, 613, 1154, 705], [1173, 540, 1198, 591], [ 619, 198, 697, 267], [ 72, 219, 288, 297], [ 401, 1109, 423, 1129], [ 179, 716, 304, 813], [ 379, 389, 400, 442], [ 638, 1084, 752, 1130], [ 25, 1004, 115, 1031], [ 247, 1154, 370, 1182], [ 375, 262, 549, 284], [ 0, 199, 40, 238], [ 383, 1142, 502, 1182], [ 396, 1120, 468, 1159], [ 622, 256, 745, 285], [ 552, 935, 586, 1051], [ 375, 676, 390, 762], [ 43, 182, 125, 263], [1159, 592, 1253, 618], [ 375, 622, 389, 678]], dtype=int32), 'class_ids': array([3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

ACHIEVED BEHAVIOUR [{'rois': array([[ 244, 1181, 245, 1182], [ 248, 1181, 249, 1182], [ 240, 1181, 242, 1182], ..., [ 267, 551, 268, 568], [ 267, 576, 268, 594], [ 259, 779, 260, 899]], dtype=int32), 'class': array([3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 2, 3, 3, 3, 3, 3, 3, 3, 2, 3, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 9, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 3, 4, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 3, 3, 2, 3, 3, 3, 3, 3, 4, 3, 4, 2, 4, 3, 4, 4, 4, 3, 3, 4, 3, 3, 4, 4, 3, 3, 3, 3, 3, 4, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 2, 3, 4, 3, 4, 3, 4, 4, 4, 3, 3, 3, 3, 2, 3, 4, 2, 4, 3, 4, 4, 2, 3, 3, 2, 3, 2, 2, 9, 4, 4, 2, 3, 2, 2, 4, 3, 2, 2, 3, 3, 2, 2, 4, 2, 3, 4, 2, 2, 9, 4, 3, 4, 9, 3, 4, 4, 3, 4, 4, 3, 2, 2, 3, 2, 4, 2, 4, 4, 3, 3, 4, 4, 4, 2, 2, 2, 3]

-> as you can see i obtain much more predictions (and they are all wrong)

[bendangnuksung]

@adebiasio I am obtaining the same (wrong) results. I wonder if there is anything to do with the conversion of the model!?

[bendangnuksung]

@adebiasio predictions are correct now. Made a script to convert to Frozen Graph PB and serving model ready for tensorflow model server. CODE

[adebiasio-estilos]

@bendangnuksung Thank you, I achieved the same results with the same strategy just this morning! Now the next step is to embed a base64ToImageInput tensor layer to make the serving more efficient!

[adebiasio-estilos]

We could follow this strategy https://github.com/tmlabonte/tendies/blob/master/minimum_working_example/tendies-basic-tutorial.ipynb

[adebiasio-estilos]

@bendangnuksung Have you tried to make a POST request to the model deployed in tf serving? It gives me: {'error': 'Incompatible shapes: [1,0,22] vs. [4]\n\t [[{{node import/mrcnn_detection/sub_1}}]]'}

Below my request:

import requests

payload = { "instances" : [{ 'import/input_image:0': molded_images.tolist(), 'import/input_image_meta:0': image_metas.tolist(), 'import/input_anchors:0': anchors.tolist() }] }

    r = requests.post('http://localhost:8501/v1/models/mask_rcnn:predict', json=payload)
    pred = json.loads(r.content.decode('unicode-escape'), strict=False)
    print(pred)

[bendangnuksung]

@adebiasio Can you try this. Made a repo instead as there were too many files conflicting.

[adebiasio-estilos]

With yours I reach the same results with ForwardModel, but actually I have not been able to make the HTTP request. Have you tried ?

[bendangnuksung]

Glad to hear about the results. I have been working with GRPC and it works, not yet tried with REST API.

[niesen1990]

@bendangnuksung Hi，I have some problems, can u help me ?
tf serving response : ....... must only have keys: input_anchors,input_image,input_image_meta"

my parms : payload = { "instances": [ {'input_anchors': anchors.tolist()}, {'input_image': molded_images.tolist()}, {'input_image_meta': image_metas.tolist()} ] }

[niesen1990]

@bendangnuksung i have this problem : { "error": "The first dimension of paddings must be the rank of inputs[4,2] [1,1,1024,1024,3] [[{{node zero_padding2d_1/Pad}}]]" }

thanks

[bendangnuksung]

@niesen1990 I assume that you were able to convert to serving model. Are you calling the model using REST API or GRPC? would you mind showing the code?

[vikiQiu]

@adebiasio predictions are correct now. Made a script to convert to Frozen Graph PB and serving model ready for tensorflow model server. CODE

Hi, your code link 404. Can you share another link? Thanks!

[janismdhanbad]

@bendangnuksung Have you tried to make a POST request to the model deployed in tf serving? It gives me: {'error': 'Incompatible shapes: [1,0,22] vs. [4]\n\t [[{{node import/mrcnn_detection/sub_1}}]]'}

Below my request:

import requests

payload = { "instances" : [{ 'import/input_image:0': molded_images.tolist(), 'import/input_image_meta:0': image_metas.tolist(), 'import/input_anchors:0': anchors.tolist() }] }

    r = requests.post('http://localhost:8501/v1/models/mask_rcnn:predict', json=payload)
    pred = json.loads(r.content.decode('unicode-escape'), strict=False)
    print(pred)

@adebiasio , were you able to resolve this error?

[moganesyan]

@bendangnuksung @adebiasio @janismdhanbad

Have you guys managed to query the REST API? I am having the same issue of incompatible shapes for import/mrcnn_detection/sub_1.

[janismdhanbad]

Hi @moganesyan , Yes I managed to query the REST API. One thing which I noticed while converting the .h5 file to .pb was that the dimensions of inputs were not specified explicitly i.e. input_images was of shape (-1,-1,-1,3), input_anchors was of shape (-1, -1, 4) and input_image_meta was of the shape (-1, -1). This was giving me an issue with shapes. I had to convert the .h5 model to the .pb model again by giving the exact input size of all the vectors leaving the first dimension. Their shape would be something like input_images --> (-1, 512, 512, 3), input_anchors --> (-1, 65472, 4), input_image_meta -->(-1, 22). These numbers would be specific to your model. You can also use saved_model_cli for checking the shapes of inputs and outputs of your model.

[moganesyan]

@janismdhanbad

Thanks for getting back to me, and I noticed the shape issue too! Glad to hear that that's the issue and not something more serious. Regarding the custom tensor shapes, where exactly do I set them? I understand that its in the function that converts the h5 to saved model, but I am unsure where exactly I should specify the tensor shapes. I am using @bendangnuksung 's code for converting the h5 to pd.

[janismdhanbad]

@moganesyan , sorry for such a delayed reply. You can change the shape inside MaskRCNN class present in model.py file. There is a variable called input_image that has a shape of [None, None, config.IMAGE_SHAPE[2]]. I simply changed it to [512, 512, config.IMAGE_SHAPE[2]] and ran the script for converting .h5 to .pb. Everything ran smoothly then.

[bendangnuksung]

Its late but if it helps I have added RESTAPI and GRPC calls in https://github.com/bendangnuksung/mrcnn_serving_ready

[moganesyan]

Not sure if people here still need it, but here's a repo of mine: https://github.com/moganesyan/tensorflow_model_deployment

It has code to export, serve and test the deployment locally (REST and GRPC). Jupyter notebooks provided

Hope it helps!

Thanks @bendangnuksung , your example helped me a lot

[ZouJiu1]

convert-to pb file

#!encoding=utf-8
'''
#-----------------
Authors:邹九
Time:2019-11-21
#-----------------
'''
"""
Copyright (c) 2019, by the Authors: Amir H. Abdi
This script is freely available under the MIT Public License.
Please see the License file in the root for details.

The following code snippet will convert the keras model files
to the freezed .pb tensorflow weight file. The resultant TensorFlow model
holds both the model architecture and its associated weights.
"""

import tensorflow as tf
from tensorflow.python.framework import graph_util
from tensorflow.python.framework import graph_io
from pathlib import Path
from absl import app
from absl import flags
from absl import logging
from mrcnn import model as modellib
from mrcnn.config import Config
import keras
import os
from keras import backend as K
from keras.models import model_from_json, model_from_yaml
from keras.utils.vis_utils import plot_model

COCO_MODEL_PATH = r'../logs/shapes20191113T1540_mask_rcnn_shapes_0199.h5'

K.set_learning_phase(0)
FLAGS = flags.FLAGS

flags.DEFINE_string('input_model', default=r'', help='Path to the input model.')
flags.DEFINE_string('input_model_json', None, 'Path to the input model '
                                              'architecture in json format.')
flags.DEFINE_string('input_model_yaml', None, 'Path to the input model architecture in yaml format.')
flags.DEFINE_string('output_model', default=r'./shapes20191113T1540_mask_rcnn_shapes_0199.pb', help='Path where the converted model will be stored.')
flags.DEFINE_boolean('save_graph_def', False,
                     'Whether to save the graphdef.pbtxt file which contains '
                     'the graph definition in ASCII format.')
flags.DEFINE_string('output_nodes_prefix', None,
                    'If set, the output nodes will be renamed to '
                    '`output_nodes_prefix`+i, where `i` will numerate the '
                    'number of of output nodes of the network.')
flags.DEFINE_boolean('quantize', False,
                     'If set, the resultant TensorFlow graph weights will be '
                     'converted from float into eight-bit equivalents. See '
                     'documentation here: '
                     'https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/graph_transforms')
flags.DEFINE_boolean('channels_first', False,
                     'Whether channels are the first dimension of a tensor. '
                     'The default is TensorFlow behaviour where channels are '
                     'the last dimension.')
flags.DEFINE_boolean('output_meta_ckpt', False,
                     'If set to True, exports the model as .meta, .index, and '
                     '.data files, with a checkpoint file. These can be later '
                     'loaded in TensorFlow to continue training.')

flags.mark_flag_as_required('input_model')
flags.mark_flag_as_required('output_model')

def load_model(input_model_path, input_json_path=None, input_yaml_path=None):
    if not Path(input_model_path).exists():
        raise FileNotFoundError(
            'Model file `{}` does not exist.'.format(input_model_path))
    try:
        model = keras.models.load_model(input_model_path)
        return model
    except FileNotFoundError as err:
        logging.error('Input mode file (%s) does not exist.', FLAGS.input_model)
        raise err
    except ValueError as wrong_file_err:
        if input_json_path:
            if not Path(input_json_path).exists():
                raise FileNotFoundError(
                    'Model description json file `{}` does not exist.'.format(
                        input_json_path))
            try:
                model = model_from_json(open(str(input_json_path)).read())
                model.load_weights(input_model_path)
                return model
            except Exception as err:
                logging.error("Couldn't load model from json.")
                raise err
        elif input_yaml_path:
            if not Path(input_yaml_path).exists():
                raise FileNotFoundError(
                    'Model description yaml file `{}` does not exist.'.format(
                        input_yaml_path))
            try:
                model = model_from_yaml(open(str(input_yaml_path)).read())
                model.load_weights(input_model_path)
                return model
            except Exception as err:
                logging.error("Couldn't load model from yaml.")
                raise err
        else:
            logging.error(
                'Input file specified only holds the weights, and not '
                'the model definition. Save the model using '
                'model.save(filename.h5) which will contain the network '
                'architecture as well as its weights. '
                'If the model is saved using the '
                'model.save_weights(filename) function, either '
                'input_model_json or input_model_yaml flags should be set to '
                'to import the network architecture prior to loading the '
                'weights. \n'
                'Check the keras documentation for more details '
                '(https://keras.io/getting-started/faq/)')
            raise wrong_file_err

class ShapesConfig(Config):
    """Configuration for training on the toy shapes dataset.
    Derives from the base Config class and overrides values specific
    to the toy shapes dataset.
    """
    # Give the configuration a recognizable name
    NAME = "shapes"

    # Number of classes (including background)
    NUM_CLASSES = 1 + 14  # background + 15 object
    # Choose the number of GPU devices
    # os.environ['CUDA_VISIBLE_DEVICES'] = '0'

    # Use small images for faster training. Set the limits of the small side
    # the large side, and that determines the image shape.
    IMAGE_RESIZE_MODE = "square"
    IMAGE_MAX_DIM = 896

    RPN_ANCHOR_SCALES = (8 * 6, 16 * 6, 32 * 6, 64 * 6, 128 * 6)  # anchor side in pixels
    # RPN_ANCHOR_SCALES = (8*5, 16*5, 32*5, 64*5, 128*5)  # anchor side in pixels

    # Reduce training ROIs per image because the images are small and have
    # few objects. Aim to allow ROI sampling to pick 33% positive ROIs.
    TRAIN_ROIS_PER_IMAGE = 100

    # Use a small epoch since the data is simple
    # STEPS_PER_EPOCH = 1000
    STEPS_PER_EPOCH = 1000

    # use small validation steps since the epoch is small
    VALIDATION_STEPS = 25

def main(args):
    # If output_model path is relative and in cwd, make it absolute from root
    output_model = FLAGS.output_model
    if str(Path(output_model).parent) == '.':
        output_model = str((Path.cwd() / output_model))

    output_fld = Path(output_model).parent
    output_model_name = Path(output_model).name
    output_model_stem = Path(output_model).stem
    output_model_pbtxt_name = output_model_stem + '.pbtxt'

    # Create output directory if it does not exist
    Path(output_model).parent.mkdir(parents=True, exist_ok=True)

    if FLAGS.channels_first:
        K.set_image_data_format('channels_first')
    else:
        K.set_image_data_format('channels_last')

    # model = load_model(FLAGS.input_model, FLAGS.input_model_json, FLAGS.input_model_yaml)
    ##--------------------------------------------------------------------------------------#
    config = ShapesConfig()
    config.display()
    MODEL_DIR = r'E:\Desktop\Projects\Mask_RCNN-master\logs'
    model = modellib.MaskRCNN(mode="inference", config=config,\
                              model_dir=MODEL_DIR)
    model.load_weights(COCO_MODEL_PATH, by_name=True)#exclude=["mrcnn_class_logits", "mrcnn_bbox_fc",\
                                # "mrcnn_bbox", "mrcnn_mask"])
    # print(model.summary())
    # plot_model(model, to_file='model1.png', show_shapes=True)
    # model_json = model.to_json()
    # with open(r'./modle.json', 'w') as file:
    #     file.write(model_json)

    print('loaded model and saved json file')
    ##--------------------------------------------------------------------------------------#
    # TODO(amirabdi): Support networks with multiple inputs
    # orig_output_node_names = [node.op.name for node in model.outputs]
    orig_output_node_names = ['mrcnn_detection/Reshape_1', 'mrcnn_class/Softmax', 'mrcnn_bbox/Reshape',\
                              'mrcnn_mask/Sigmoid', 'ROI/packed_2', 'rpn_class/concat', 'rpn_bbox/concat']

    if FLAGS.output_nodes_prefix:
        num_output = len(orig_output_node_names)
        pred = [None] * num_output
        converted_output_node_names = [None] * num_output

        # Create dummy tf nodes to rename output
        for i in range(num_output):
            converted_output_node_names[i] = '{}{}'.format(
                FLAGS.output_nodes_prefix, i)
            pred[i] = tf.identity(model.outputs[i],
                                  name=converted_output_node_names[i])
    else:
        converted_output_node_names = orig_output_node_names
    logging.info('Converted output node names are: %s',
                 str(converted_output_node_names))

    sess = K.get_session()
    if FLAGS.output_meta_ckpt:
        saver = tf.train.Saver()
        saver.save(sess, str(output_fld / output_model_stem))

    if FLAGS.save_graph_def:
        tf.train.write_graph(sess.graph.as_graph_def(), str(output_fld),
                             output_model_pbtxt_name, as_text=True)
        logging.info('Saved the graph definition in ascii format at %s',
                     str(Path(output_fld) / output_model_pbtxt_name))

    if FLAGS.quantize:
        from tensorflow.tools.graph_transforms import TransformGraph
        transforms = ["quantize_weights", "quantize_nodes"]
        transformed_graph_def = TransformGraph(sess.graph.as_graph_def(), [],
                                               converted_output_node_names,
                                               transforms)
        constant_graph = graph_util.convert_variables_to_constants(
            sess,
            transformed_graph_def,
            converted_output_node_names)
    else:
        constant_graph = graph_util.convert_variables_to_constants(
            sess,
            sess.graph.as_graph_def(),
            converted_output_node_names)

    graph_io.write_graph(constant_graph, str(output_fld), output_model_name,
                         as_text=False)
    logging.info('Saved the freezed graph at %s',
                 str(Path(output_fld) / output_model_name))

if __name__ == "__main__":
    app.run(main)

load pb model

def load_detection_model(model):
    config = tf.ConfigProto()
    config.gpu_options.allow_growth = True
    detection_graph = tf.Graph()
    with detection_graph.as_default():
        od_graph_def = tf.GraphDef()
        with tf.gfile.GFile(model, 'rb') as fid:
            serialized_graph = fid.read()
            od_graph_def.ParseFromString(serialized_graph)
            tf.import_graph_def(od_graph_def, name='')
        input_image = tf.get_default_graph().get_tensor_by_name('input_image:0')
        input_image_meta = tf.get_default_graph().get_tensor_by_name('input_image_meta:0')
        input_anchors = tf.get_default_graph().get_tensor_by_name('input_anchors:0')
        detections = tf.get_default_graph().get_tensor_by_name('mrcnn_detection/Reshape_1:0')
        mrcnn_mask = tf.get_default_graph().get_tensor_by_name('mrcnn_mask/Sigmoid:0')
    sessd=tf.Session(config=config,graph=detection_graph)
    print('Loaded detection model from file "%s"' % model)
    return sessd, input_image, input_image_meta, input_anchors, detections, mrcnn_mask

sessd, input_image, input_image_meta, input_anchors, detections, mrcnn_mask = load_detection_model(model_path)
results = model.detect_pb([image], sessd, input_image, input_image_meta, input_anchors, detections, mrcnn_mask,verbose=1)

use model, add to mrcnn/model.py

    def detect_pb(self, images, sessd, input_image, input_image_meta, input_anchors, detections, mrcnn_mask, verbose=1):
        """Runs the detection pipeline.

        images: List of images, potentially of different sizes.

        Returns a list of dicts, one dict per image. The dict contains:
        rois: [N, (y1, x1, y2, x2)] detection bounding boxes
        class_ids: [N] int class IDs
        scores: [N] float probability scores for the class IDs
        masks: [H, W, N] instance binary masks
        """
        assert self.mode == "inference", "Create model in inference mode."
        assert len(
            images) == self.config.BATCH_SIZE, "len(images) must be equal to BATCH_SIZE"

        # if verbose:
        #     log("Processing {} images".format(len(images)))
        #     for image in images:
        #         log("image", image)

        # Mold inputs to format expected by the neural network
        molded_images, image_metas, windows = self.mold_inputs(images)

        # Validate image sizes
        # All images in a batch MUST be of the same size
        image_shape = molded_images[0].shape
        # print(image_shape, molded_images.shape)
        for g in molded_images[1:]:
            assert g.shape == image_shape,\
                "After resizing, all images must have the same size. Check IMAGE_RESIZE_MODE and image sizes."

        # Anchors
        anchors = self.get_anchors(image_shape)
        # Duplicate across the batch dimension because Keras requires it
        # TODO: can this be optimized to avoid duplicating the anchors?
        anchors = np.broadcast_to(anchors, (self.config.BATCH_SIZE,) + anchors.shape)

        # if verbose:
        #     log("molded_images", molded_images)
        #     log("image_metas", image_metas)
        #     log("anchors", anchors)
        # Run object detection
        # detections, _, _, mrcnn_mask, _, _, _ =\
        #     self.keras_model.predict([molded_images, image_metas, anchors], verbose=0)
        detectionsed, mrcnn_masked = sessd.run([detections, mrcnn_mask], feed_dict = {input_image: molded_images, \
                                                               input_image_meta: image_metas, \
                                                               input_anchors: anchors})
        mrcnn_masked = np.expand_dims(mrcnn_masked, 0)
        detections = np.array(detectionsed)
        mrcnn_mask = np.array(mrcnn_masked)
        # Process detections
        results = []
        for i, image in enumerate(images):
            xi = detections[i]
            yi = mrcnn_mask[i]
            moldedi = molded_images[i]
            windowsi = windows[i]
            final_rois, final_class_ids, final_scores, final_masks =\
                self.unmold_detections(detections[i], mrcnn_mask[i],
                                       image.shape, molded_images[i].shape,
                                       windows[i])
            results.append({
                "rois": final_rois,
                "class_ids": final_class_ids,
                "scores": final_scores,
                "masks": final_masks,
            })
        return results