How to convert Mask rcnn model to Tensorflow .pb

[luoshanwei]

I want to run the mask rcnn on android , but I have not .pb file

[fastlater]

@ericj974 Will you pull your script? I guess it will be of interest for several users who would like to serve their trained models.

@liangbo-1 As I mentioned before, code is doing fine without error. You got no error, you got a warning. Use the print function to check what the script outputs every few lines.

[ypflll]

Hi, I tried to use this code to run the .pb model file which I get using @ericj974 's code and follow the step @chenyuZha mentioned above:

output_graph_path = r"./mask_rcnn.pb"
with tf.Session() as sess:
    tf.global_variables_initializer().run()
    output_graph_def = tf.GraphDef()
    with open(output_graph_path, "rb") as f:
        output_graph_def.ParseFromString(f.read())
        _ = tf.import_graph_def(output_graph_def, name="")

    image = cv2.imread('/xxx/a.jpg')
    image = cv2.resize(image,(256,256))
    image = np.expand_dims(image,0)

    image_resized, window, scale,_ = resize_image(image, min_dim=None, max_dim=None, padding=False)
    image_meta = compose_image_meta(image_id=1, image_shape=image_resized.shape, window=window, active_class_ids=[0,1])
    image_meta = np.expand_dims(image_meta,0)
    graph = tf.get_default_graph()
    input_image = graph.get_tensor_by_name("input_image:0")
    input_image_meta = graph.get_tensor_by_name("input_image_meta:0")
    mrcnn_mask = graph.get_tensor_by_name("output_mrcnn_mask:0")

    #[detections, mrcnn_class, mrcnn_bbox, mrcnn_mask,rois, rpn_class, rpn_bbox]
    feed = {input_image:image_resized, input_image_meta:image_meta}
    result = sess.run(mrcnn_mask, feed_dict = feed)
    print(result.shape)

But I get a problem: I can only get a tensor in 'trainning' mode, (that is the code below: if mode == "training" ). When I tried to get a tensor in 'inference' mode using code like: KL.Lambda(lambda x: x * 1, name="output_rois")(rois) or tf.identity, it always raises an error: KeyError: "The name 'mask:0' refers to a Tensor which does not exist. The operation, 'mask', does not exist in the graph."

So I need to input a para that tells the model to do 'inference' mode? But how?

[fastlater]

@ypflll I dont know what is wrong with your code but you can follow this script. I will try to pull this after a few changes. Any suggestion about the code and how to make it looks cleaner is welcome. [Update 3/26]: A new and more clear notebook link

I am waiting for @ericj974 because he wrote the first part, model serving and this could be considered as a second part, infering from a pb.

[ypflll]

@fastlater Good job. I'll try it.

[chenyuZha]

@fastlater Thanks for your code. I tested it and it works well. Just a little question: I noticed that your use library scipy to do the pre-processing of images. (scipy.misc.imresize). I changed it to cv2.resize, and I obtained the masks which are little bit different( just the border), and the scores of each object are also different, I compare these 2 results and found that cv2.resize, en general, give a better detection scores.. As I used my own dataset, I don't know if it is the same case of yours...

[fastlater]

@ypflll You are welcome. Let me know if you get any trouble running the script. @chenyuZha Thank you for your feedback. Actually, I just reused the resize function from the original utils.py. However, if you already tested both functions (scipy.misc.imresize and cv2.resize) and found cv2.resize function was giving better results, it sounds interesting.

[ypflll]

@fastlater Hi, your code works well. But I have a problem with the time consuming. When predicting more than one pictures with size of 320*320, on a gpu, the first pic takes more than 60s!!! But the next pics only takes less than 300ms.

Have you tested your time cost and met this problem?

[fastlater]

@ypflll well, first image always take long than the rest. The slowness in the first run probably stems from things like device memory copying, cache misses, etc. As I said, the functions related to calculate, unmold and display results are taken from the demo. This problem also happens when you infer loading the model from weights, and not from pb. I have no idea if there is a way to speed up the first run, I hope it will.

I haven't calculate the time cost but my first run takes less than 60 seconds for sure.

[ypflll]

@fastlater I just measure the time cost of self.keras_model.predict(), so I don't think the problem relates to the pre-process or the post-process. Time cost depends on your image size, I use 320*320 and get this result.

I also tested in an old version code(1c51787), the first pic only takes 1.9s. So, I think there is something with the last few commits. Maybe this is beyond this issue, I'll open another one.

[fastlater]

@ypflll Yes, open a new thread will be better. As I said, I noticed that even when testing the demo code so something is making the first run slower.

[githubXiaosong]

@fastlater Thank you for sharing your script. but I've used your script to make the following error: Tensor("input_image:0", shape=(?, 1024, 1024, 3), dtype=float32) Tensor("input_image_meta:0", shape=(?, ?), dtype=float32) Found Tensor("output_detections:0", dtype=float32) Found Tensor("output_mrcnn_class:0", shape=(?, 1000, 81), dtype=float32) Found Tensor("output_mrcnn_bbox:0", shape=(?, 1000, 81, 4), dtype=float32) Found Tensor("output_mrcnn_mask:0", shape=(?, 100, 28, 28, 81), dtype=float32) Found Tensor("output_rois:0", shape=(1, ?, ?), dtype=float32) Windows (1, 4) [[ 0 172 1024 852]] 2018-03-22 17:06:46.632334: W tensorflow/core/framework/op_kernel.cc:1192] Unknown: KeyError: 'pyfunc_0' @fastlater Can you give me some advice? thank you .

[liangbo-1]

@ypflll Do you use mask_rcnn_coco.h5 to get .pb? i use the mask_rcnn_coco.h5 to get .pb and then,when i used @fastlater' script to test the .pb file, I had the following errors. Windows (1, 4) [[ 0 172 1024 852]] 2018-03-22 19:48:29.487264: W tensorflow/core/framework/op_kernel.cc:1192] Unknown: KeyError: 'pyfunc_0' Could you give me some advices?thank you

[ypflll]

@liangbo-1 No. I think you should train your own model before convert.

[liangbo-1]

@ypflll

Now I use my own dataset to train the model, convert it into .pb, and then use the script, but still have the same error. Is it related to the size of the picture? What size is the picture you use?

[ypflll]

@liangbo-1 I think your code is not the newest.

[ypflll]

@fastlater Hi, I am trying to convert your code to c++. It's a little complicated to call sess->run() 5 times to get the raw results and then post process to get the final results. Do you think the post-processing can be done inside the model so we can get the results(masks and rois coordinates) just call sess->run() one time? Another question: I am not very clear about how to get the tensor by name. Like: detectionsT = sess.graph.get_tensor_by_name('output_detections:0'). Actually, in the model, there's no tensor named 'output_detections'. So what's the rule to get it?

[fastlater]

@ypflll Thank you for your feedback.

1. From the model.py

   detections, mrcnn_class, mrcnn_bbox, mrcnn_mask, \
           rois, rpn_class, rpn_bbox =\
           self.keras_model.predict([molded_images, image_metas], verbose=0)

   you can notice that the original code outputs each result separately. Since I haven't edit the original model, I had to use sess.run several times. If you know more about tensorflow and keras, maybe you can propose a way to "encapsulate/group/Concatenate" all the results into a single output tensor.

About the names of the outputs (for example, output_detections), it is the name defined in the exporting demo script. Feel free to follow my jupyter notebook demo where I put together both codes (exporting script prepared by @ericj974 and my importing script) link . This is the same code used before but I don't know why, I get error in sess = tf.session() in jupyter. I hav been trying to solve the problem so I changed that code line to tf.InteractiveSession() (based on some webpages) but still get error. If someone can make it run properly, let me know what do I need to change. At the end, this issue was created to share, discuss and help each other.

@githubXiaosong @liangbo-1 let me know if you found a way to solve that error. In my side, the script was running without problem and I cannot reproduce that error so I dont know what to say about it. I hope you can find the solution and share the solution with the rest of the community. Feel free to test the jupyter notebook mentioned in this comment too.

[ivshli]

@fastlater @ypflll
Hi, there is a code example about to convert code into C++ write by @moorage which you need only call it once and can get all outputs, I haven't test it yet, but your are very welcome to test and give the feedback about it, let's make the community more active :) See code here: https://github.com/matterport/Mask_RCNN/issues/222#issuecomment-373130661

[ypflll]

@fastlater Still not clear why tensor name like 'outputdetections' not defined in the model but you can get it. I guess keras add a prefix 'output' automatically here: model = KM.Model([input_image, input_image_meta], [detections, mrcnn_class, mrcnn_bbox, mrcnn_mask, rpn_rois, rpn_class, rpn_bbox], name='mask_rcnn')

@ivshli I'll test it.

[moorage]

@ypflll if you look at the export to .pb code here: https://github.com/ericj974/Mask_RCNN/blob/master/scripts/export_model.py#L38

It prepends output_ to each node.

[ypflll]

@moorage That's exactly what I want!!

[samhodge]

OK I have been hacking away with @moorage 's source over here: https://github.com/samhodge/OpenCVTensorflowExample/blob/master/OpenCVTensorflowExample

but putting in @moorage 's snippet from https://github.com/matterport/Mask_RCNN/issues/222

it is currently commented out but when it does run

Currently it is crashing at: https://github.com/samhodge/OpenCVTensorflowExample/blob/master/OpenCVTensorflowExample/main.cpp#L258

Clearly from the attached screenshot the dimensions are:

not

[1,100,28,28,2]

but [1,100,28,28,81]

I assume the 81 refers to the number classifications, but

So how do we get from the 81 that we have to the 2 that @moorage's snippet is expecting?

from https://github.com/matterport/Mask_RCNN/issues/222

Anyway my saved model is here: https://drive.google.com/open?id=1xMqHddE14dWPi6PreyjqQg_hT_CNntsw

Has anyone else made progress getting this to run in C++?

Sam

[moorage]

@samhodge I only had 2 channels because I only trained two classes -- it's a good point that you brought that up. You'll have to change my code to expect the number of classes in your trained system.

In particular, these lines will have to change: https://github.com/samhodge/OpenCVTensorflowExample/blob/master/OpenCVTensorflowExample/main.cpp#L277-L279

[samhodge]

Thanks for your time @moorage.

I will see what I can mix up

[ypflll]

@liangbo-1 I met this when I am using an old version code. Try to pull the newest code.

[liangbo-1]

@ypflll Thank you for your reply. Can you share your newest code? Thank you

[ypflll]

@liangbo-1 I mean the newest code of this repo.

[AliceDinh]

I have same error with @Surmeh.

Using TensorFlow backend.
Input file specified (mask_rcnn_mymodel.h5) only holds the weights, and not the model definition.
    Save the model using mode.save(filename.h5) which will contain the network architecture
    as well as its weights.

    If the model is saved using model.save_weights(filename.h5), the model architecture is
    expected to be saved separately in a json format and loaded prior to loading the weights.
    Check the keras documentation for more details (https://keras.io/getting-started/faq/)

Traceback (most recent call last):
  File "keras_to_tensorflow.py", line 123, in <module>
    raise err
  File "keras_to_tensorflow.py", line 114, in <module>
    net_model = load_model(weight_file_path)
  File "C:\Users\Alice\Anaconda3\lib\site-packages\keras\models.py", line 241, in load_model
    raise ValueError('No model found in config file.')
ValueError: No model found in config file.

So I reckon that, I have to modify my code at model.py by: mode.save(filename.h5) instead of model.save_weights(filename.h5), but I could see only this line relevant:

        callbacks = [
            keras.callbacks.TensorBoard(log_dir=self.log_dir,
                                        histogram_freq=0, write_graph=True, write_images=False),
            keras.callbacks.ModelCheckpoint(self.checkpoint_path,
                                            verbose=0, save_weights_only=True),
        ]

I should change: save_weights_only=False, right? Pleas help

[ypflll]

@liangbo-1 Didn't meet this. Maybe you can load and run your .h5 file before convert to .pb. Also you may check the inference_config.

[AliceDinh]

@Adriel-M and @Cpruce: Please please show me how did you fix the error: ValueError: Unknown layer: BatchNorm Thanks

[AliceDinh]

@ericj974 : Please help me here. I run your script and here is the way I did it: https://github.com/ericj974/Mask_RCNN/tree/master/scripts

[1] I copies codes inside the function def export() and put under the evaluation part. [2] At the end of evaluation part, I put model.keras_model.save(filename) with filename = 'mask_rcnn.pb' hardcoded. [3] I also provide the model_filepath = os.path.join(ROOT_DIR, "models/mask_rcnn_mymodel.h5").

And it is working like a charm, the output is mask_rcnn.pb but I reckoned that the size of this .pb file is exactly the same as original .h5 file. I really confused here that the process of converting is successfully or not.

[ypflll]

@liangbo-1 I use this: https://github.com/ericj974/Mask_RCNN/blob/master/scripts/export_model.py

[liangbo-1]

@fastlater @ypflll I used your script: https://github.com/fastlater/Mask_RCNN/blob/master/demo_export_import_model.ipynb and got the error: /usr/local/lib/python3.5/dist-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters Using TensorFlow backend. /home/lb/fastlater-MASK-RCNN/Mask_RCNN-master/logs /home/lb/fastlater-MASK-RCNN/Mask_RCNN-master/logs/ec20180321T0844/mask_rcnn_ec_0045.h5 2018-04-08 19:36:48.102086: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA 2018-04-08 19:36:48.196196: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:892] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2018-04-08 19:36:48.196454: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties: name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.683 pciBusID: 0000:01:00.0 totalMemory: 7.92GiB freeMemory: 7.48GiB 2018-04-08 19:36:48.196468: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1) Model loaded. ['output_detections', 'output_mrcnn_class', 'output_mrcnn_bbox', 'output_mrcnn_mask', 'output_rois', 'output_rpn_class', 'output_rpn_bbox'] Converted 690 variables to const ops. Saving frozen graph mask_rcnn_New.pb ... 3364 ops in the frozen graph. Graph loaded. 2018-04-08 19:36:52.382194: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1) Image loaded. Processing 1 images image shape: (640, 1024, 3) min: 0.00000 max: 255.00000 RGB image loaded and preprocessed. IMAGE_PADDING: True (640, 1024, 3) Image resized at: (1024, 1024, 3) (192, 0, 832, 1024) 1 Image molded Meta of image prepared (1, 1024, 1024, 3) Images meta: [[ 0 640 1024 3 192 0 832 1024 0 0 0]] Tensor("input_image:0", shape=(?, 1024, 1024, 3), dtype=float32) Tensor("input_image_meta:0", shape=(?, ?), dtype=float32) Found Tensor("output_detections:0", dtype=float32) Found Tensor("output_mrcnn_class:0", shape=(?, 1000, 3), dtype=float32) Found Tensor("output_mrcnn_bbox:0", shape=(?, 1000, 3, 4), dtype=float32) Found Tensor("output_mrcnn_mask:0", shape=(?, 100, 28, 28, 3), dtype=float32) Found Tensor("output_rois:0", shape=(1, ?, ?), dtype=float32) Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1323, in _do_call return fn(*args) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1302, in _run_fn status, run_metadata) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value conv1/kernel [[Node: conv1/kernel/read = IdentityT=DT_FLOAT, _class=["loc:@conv1/kernel"], _device="/job:localhost/replica:0/task:0/device:GPU:0"]] [[Node: output_detections/_79 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_4323_output_detections", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "infere_from_pb_new.py", line 315, in detections = sess.run(detectionsT, feed_dict={img_ph: molded_images, img_meta_ph: image_metas}) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 889, in run run_metadata_ptr) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1120, in _run feed_dict_tensor, options, run_metadata) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1317, in _do_run options, run_metadata) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1336, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value conv1/kernel [[Node: conv1/kernel/read = IdentityT=DT_FLOAT, _class=["loc:@conv1/kernel"], _device="/job:localhost/replica:0/task:0/device:GPU:0"]] [[Node: output_detections/_79 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_4323_output_detections", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Caused by op 'conv1/kernel/read', defined at: File "infere_from_pb_new.py", line 90, in model_dir=train_log_dirpath) File "/home/lb/fastlater-MASK-RCNN/Mask_RCNN-master/model.py", line 1735, in init self.keras_model = self.build(mode=mode, config=config) File "/home/lb/fastlater-MASK-RCNN/MaskRCNN-master/model.py", line 1791, in build , C2, C3, C4, C5 = resnet_graph(input_image, "resnet101", stage5=True) File "/home/lb/fastlater-MASK-RCNN/Mask_RCNN-master/model.py", line 152, in resnet_graph x = KL.Conv2D(64, (7, 7), strides=(2, 2), name='conv1', use_bias=True)(x) File "/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py", line 592, in call self.build(input_shapes[0]) File "/usr/local/lib/python3.5/dist-packages/keras/layers/convolutional.py", line 138, in build constraint=self.kernel_constraint) File "/usr/local/lib/python3.5/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py", line 416, in add_weight constraint=constraint) File "/usr/local/lib/python3.5/dist-packages/keras/backend/tensorflow_backend.py", line 395, in variable v = tf.Variable(value, dtype=tf.as_dtype(dtype), name=name) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py", line 213, in init constraint=constraint) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py", line 356, in _init_from_args self._snapshot = array_ops.identity(self._variable, name="read") File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/array_ops.py", line 125, in identity return gen_array_ops.identity(input, name=name) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 2071, in identity "Identity", input=input, name=name) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2956, in create_op op_def=op_def) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1470, in init self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

FailedPreconditionError (see above for traceback): Attempting to use uninitialized value conv1/kernel [[Node: conv1/kernel/read = IdentityT=DT_FLOAT, _class=["loc:@conv1/kernel"], _device="/job:localhost/replica:0/task:0/device:GPU:0"]] [[Node: output_detections/_79 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_4323_output_detections", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Then I found that I need to add this sentence to your code: sess = tf.Session() sess.run(tf.initialize_all_variables()) and it won't get the same error. However, the final visualization results are wrong. Or it's disordered or unable to frame the location of the target object. My trained.H5 files work very well. @fastlater @ypflll Can you give me some advices? thank you

[fastlater]

@liangbo-1 If you are in hurry, try to use the notebook as a reference for your own script. I recommend you to run the script as a py file and not as jupyter notebook. I still dont know why it can run without error in python but outputs this kind of error using jupyter notebook.

[ypflll]

@liangbo-1 Maybe you can check you input again. Also, try different interpolation methods when resize.

[IgorKasianenko]

Hi, thanks for a lot of useful information! Several questions: @samhodge You shared a model.pb. What is your model dimensions? 2 or 81 classes? Does it contain person class? I'm still confused, the model that you shared is keras model in protobuf, isn't it?

And main question to all, how did you overcome "Unknown layer: BatchNorm"?

@Cpruce could you confirm that you successfully convert model.h5 with keras_to_tensorflow.py? If yes, may you be so kind to share result as samhodge did in his comment?

[samhodge]

The model is a frozen graph to run in Tensorflow it has 81 classes and includes person.

[samhodge]

@IgorKasianenko I was testing out my .pb file I put on google drive and it doesnt seem to be detecting much according to https://github.com/samhodge/OpenCVTensorflowExample/commit/d5784b6787f1fdca3a78daf732bf77221573f5c8

But I might have made a dumb error like BGR rather than RGB, if someone want to play with the code there, you dont need XCode I can upload a Makefile for Linux if I get a chance. Basically you need libtensorflow_cc.so and libtensorflow_framework.so in your LD_LIBRARY_PATH and when you link.

[praneethkalimisetty]

can someone post the video how to open and run this repository to visualize the graph in tensorboard

https://github.com/ericj974/Mask_RCNN/blob/master/scripts/export_model.py please give a video on how to open and view this file @ericj974 @Surmeh

[EscVM]

@fastlater Have you found a way to concatenate all outputs in a single output node?

[kuonangzhe]

@fastlater Hi, I just ran your code. Everything works awesome in make .pb file, but I got a weird problem during pb file inference, telling that nms is not workable on GPU. I used your code with python files instead of notebook. The error details are as follows:

InvalidArgumentError (see above for traceback): Cannot assign a device for operation 'ROI/rpn_non_max_suppression/NonMaxSuppressionV2_1': Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available.
     [[Node: ROI/rpn_non_max_suppression/NonMaxSuppressionV2_1 = NonMaxSuppressionV2[_device="/device:GPU:0"](ROI/strided_slice_21_1, ROI/strided_slice_22_1, ROI/rpn_non_max_suppression/NonMaxSuppressionV2/max_output_size_1, ROI/rpn_non_max_suppression/iou_threshold_1)]]

Does this related to the version issues, or any other suggestions? I am using tf 1.7, keras 2.1.5, CUDA 8.0. Thank you so much!

There are also two other problems, but can be dealt with work around methods. One is for input_image_meta, when there are two classes (one is background), from compose_image_meta I get shape (1, 10), but in graph it is (?, 14). Another problem is that it asks for input_anchors, which needs to add to feed_dict in session run.

[parai]

Hi all: with the https://github.com/ericj974/Mask_RCNN/blob/master/scripts/export_model.py, I sucessfully get the pb generated, but how to use it, is there any demo python script to load this pb and use it to process images and give some output just like the demo.ipynb.

PS: I add below 3 line of code to the demo.ipynb to get the pb generated.

sys.path.append(os.path.join(ROOT_DIR, "mrcnn")) from scripts.export_model import export export(config, MODEL_DIR, COCO_MODEL_PATH)

2018-05-23 18:15:52.814432: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929] 0 2018-05-23 18:15:52.814478: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0: N 2018-05-23 18:15:52.814797: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4708 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1) Loading weights from /media/data/Mask_RCNN/mask_rcnn_coco.h5 Converted 690 variables to const ops. Saving frozen graph mask_rcnn.pb ... 3712 ops in the frozen graph.

[parai]

Hi all: with some work and modification, I could run the pb now in the tensorflow. check this commit https://github.com/parai/Mask_RCNN/commit/6289c1bd08fc90a1c3e296be8155674651f82a4b

[jezelmae12400]

@parai Thank you so much for this! I just got 1 problem

KeyError: "The name 'input_anchors_1:0' refers to a Tensor which does not exist. The operation, 'input_anchors_1', does not exist in the graph."

[gustavz]

@parai did you measure/time your inference pipeline? how long does the processing of one image take (i mean the whole loop, not just the detection part)? Furthermore You are creating a session for each output, why not combining them all into one single session with one single output dictionary?

[parai]

@GustavZ Currently I am a new begainer in the AI, I just try to bring the code of others up for the 2.0 release. And I didn't do any performance test.

[parai]

@jezelmae12400 are you sure you are using a pb that converted from the 2.0 release mask_rcnn_coco.h5

[parai]

Hi all: I am trying to study the graph by the tensorboard, but I really can't understand the kind of shape such as shape=(?, ?, ?, 3), what it is, a 4D shape, is there any help document to introduce it, I can't find much valuable information on net. Tensor("input_image_1:0", shape=(?, ?, ?, 3), dtype=float32) Tensor("input_anchors_1:0", shape=(?, ?, 4), dtype=float32) Tensor("input_image_meta_1:0", shape=(?, 93), dtype=float32) Found Tensor("output_detections:0", shape=(1, 100, 6), dtype=float32) Found Tensor("output_mrcnn_class:0", shape=(?, 1000, 81), dtype=float32) Found Tensor("output_mrcnn_bbox:0", shape=(?, 1000, 81, 4), dtype=float32) Found Tensor("output_mrcnn_mask:0", shape=(?, 100, 28, 28, 81), dtype=float32) Found Tensor("output_rois:0", shape=(1, ?, ?), dtype=float32) Windows (1, 4) [[ 128 0 896 1024]]

[chohaku84]

hi @AliceDinh and @ericj974 https://github.com/matterport/Mask_RCNN/issues/url above link has expired but I really need help with how to save the model(not only the weights), could you repost the functions?

[ericj974]

@chohaku84 New script here