luoshanwei commented 6 years ago

I want to run the mask rcnn on android , but I have not .pb file

chohaku84 commented 6 years ago

@ericj974 Thank you very much.

xjohnxjohn commented 6 years ago

@jmtatsch are you solved the problem coco.py save the model?

jmtatsch commented 6 years ago

@xjohnxjohn yes. I can post it on friday.

VinniaKemala commented 6 years ago

@jmtatsch How did you fix ValueError: Unknown layer: BatchNorm ?

jmtatsch commented 6 years ago

@xjohnxjohn @VinniaKemala try using this gist

patrick-llgc commented 6 years ago

@liangbo-1 If you are in hurry, try to use the notebook as a reference for your own script. I recommend you to run the script as a py file and not as jupyter notebook. I still dont know why it can run without error in python but outputs this kind of error using jupyter notebook.

@liangbo-1 @fastlater Thanks for the notebook from @fastlater. It is very helpful. I figured out why there was an error in the jupyter notebook. You have to use tf.reset_default_graph() before loading the graph from pb again. Otherwise there seem to be multiple copies of the graph (as we built it from scratch earlier in the notebook), and tensorflow cannot match the tensors with the correct weights.

slothkong commented 6 years ago

@jmtatsch Quick question. Was it necessary to get rid or replace the custom 'BatchNorm' layers? I noticed you were also having problems with them.

slothkong commented 6 years ago

@fastlater You say that the reason why your code(https://github.com/fastlater/Mask_RCNN/blob/master/demo_export_import_model.ipynb) to evaluate images with the pb model yields an error is because is running inside a jupyter notebook? That is weird. I guess I will give it a try by moving the code a .py script.

patrick-llgc commented 6 years ago

@slothkong see my comment right above yours. The reason is that there are two networks in the default graph. In order to run it in the ipynb, you have to reset the default graph.

fastlater commented 6 years ago

@slothkong As @patrick-12sigma mentioned, I made a little mistake when I copied the code from the py script to the jupyter notebook. The reason I created the notebook was to illustrate more clearly the code execution since I knew the code was working fine in my side. Good to know it was a starting point for other notebooks. Whenever I have time, I will try to add the fix.

@EscVM I haven't try to do that yet. did you make it?

slothkong commented 5 years ago

@patrick-12sigma @liangbo-1 @fastlater I can confirm that adding tf.reset_default_graph() right before the call to tf.gfile.FastGFile() makes it possible to load the graph from pb.

However, I noticed that the model has 3 inputs:

Tensor("input_image:0", shape=(?, ?, ?, 3), dtype=float32)
Tensor("input_image_meta:0", shape=(?, 93), dtype=float32)
Tensor("input_anchors:0", shape=(?, ?, 4), dtype=float32)

but code in @fastlater's notebook does not put any data in the anchor's tensor. When I run the session to get detections tensorflow realizes this and spits this error:

InvalidArgumentError: You must feed a value for placeholder tensor 'input_anchors' with dtype float and shape [?,?,4]
     [[Node: input_anchors = Placeholder[dtype=DT_FLOAT, shape=[?,?,4], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

You guys don't have the same issue?

EDIT: I just realized that @parai posted useful code in this script

PreethaVeera commented 5 years ago

@chohaku84 New script here

@ericj974 I am trying to convert the mask_rcnn_coco.h5 file into tensorflow .pb format. I tried using the keras to tensorflow code. But I am facing with the issue : raise ValueError('No model found in config file.')
I tried your script still I am facing with the config file missing error. I have added the stack trace. Using TensorFlow backend.

Traceback (most recent call last): File "export.py", line 102, in main(args) File "export.py", line 34, in main config_inference = load_config_files(config_path=config_path) File "export.py", line 91, in load_config_files assert os.path.exists(config_path) File "/home/mcw/.conda/envs/py35/lib/python3.6/genericpath.py", line 19, in exists os.stat(path) TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType

How to fix this?

ericj974 commented 5 years ago

@PreethaVeera You need to (1) Create a my_config.py containing a class MyConfig that will extend Config class in config.py (1-a) add class property NAME = 'choose_a_name' (it will override the default value of None) (1-b) add class property NUM_CLASSES = 81 (it will override the default value of 1)

(2) launch the export script where you specify (2-a) model_path = path_to_h5 (2-b) log_dirpath = '' (2-c) config_path = path_to_my_config_file

=> output pb file will be created in the same folder as your h5

jbrownkramer commented 5 years ago

@ericj974 Let me start by thanking you. Your script has created a pb file from my h5 file where no other script I used was able to.

However, I didn't exactly follow the comment above. Instead, I modified a copy of config.py to update NAME and NUM_CLASSES in the Config class itself and pointed the script to it. In particular, I tried putting the following code in its own file and pointing the script to it, and it didn't work.

#This doesn't work
import mrcnn.config as config

class MyConfig(config.Config):
    NAME = "MyConfig"
    NUM_CLASSES = 81

I get the following error

C:\Python36\lib\site-packages\h5py\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Using TensorFlow backend.
Traceback (most recent call last):
  File "export.py", line 102, in <module>
    _main_(args)
  File "export.py", line 34, in _main_
    config_inference = load_config_files(config_path=config_path)
  File "export.py", line 96, in load_config_files
    config = getattr(mod, Config.__name__)
AttributeError: module 'configs_' has no attribute 'Config'

Any insight into what I'm doing wrong?

ifrogy commented 5 years ago

@jbrownkramer Try: from mrcnn.config import Config class MyConfig(Config):

javierfs commented 5 years ago

I have the following issue:

this is root_dir: /home/javier/disk_user/OCR_project/Mask_RCNN/infering
this is Image_dir: /home/javier/disk_user/OCR_project/Mask_RCNN/infering/../customImages/val/
choose image file is 490_2.jpg
(5856, 4128, 3)
Processing 1 images
image                    shape: (5856, 4128, 3)       min:    0.00000  max:  255.00000  uint8
RGB image loaded and preprocessed.
/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/skimage/transform/_warps.py:110: UserWarning: Anti-aliasing will be enabled by default in skimage 0.15 to avoid aliasing artifacts when down-sampling images.
  warn("Anti-aliasing will be enabled by default in skimage 0.15 to "
(5856, 4128, 3)
Image resized at:  (1024, 1024, 3)
(0, 151, 1024, 873)
0.17486338797814208
Image molded
Meta of image prepared
(1, 1024, 1024, 3)
anchors shape is (1, 261888, 4) float32
Tensor("input_image:0", shape=(?, ?, ?, 3), dtype=float32, device=/device:CPU:0)
Tensor("input_anchors:0", shape=(?, ?, 4), dtype=float32, device=/device:CPU:0)
Tensor("input_image_meta:0", shape=(?, 14), dtype=float32, device=/device:CPU:0)
Found  Tensor("output_detections:0", shape=(2, 100, 6), dtype=float32)
Found  Tensor("output_mrcnn_class:0", shape=(?, 1000, 2), dtype=float32)
Found  Tensor("output_mrcnn_bbox:0", shape=(?, 1000, 2, 4), dtype=float32)
Found  Tensor("output_mrcnn_mask:0", shape=(2, 100, 28, 28, 2), dtype=float32)
Found  Tensor("output_rois:0", shape=(2, ?, ?), dtype=float32)
Windows (1, 4)   [[   0  151 1024  873]]
Traceback (most recent call last):
  File "infere_from_pb.py", line 318, in <module>
    tf.app.run()
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "infere_from_pb.py", line 78, in main
    img_anchors_ph: image_anchors})
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
    run_metadata_ptr)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run
    str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (1, 93) for Tensor 'input_image_meta:0', which has shape '(?, 14)'

Anyone knows why I got this?

javierfs commented 5 years ago

I have the following issue:

this is root_dir: /home/javier/disk_user/OCR_project/Mask_RCNN/infering
this is Image_dir: /home/javier/disk_user/OCR_project/Mask_RCNN/infering/../customImages/val/
choose image file is 490_2.jpg
(5856, 4128, 3)
Processing 1 images
image                    shape: (5856, 4128, 3)       min:    0.00000  max:  255.00000  uint8
RGB image loaded and preprocessed.
/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/skimage/transform/_warps.py:110: UserWarning: Anti-aliasing will be enabled by default in skimage 0.15 to avoid aliasing artifacts when down-sampling images.
  warn("Anti-aliasing will be enabled by default in skimage 0.15 to "
(5856, 4128, 3)
Image resized at:  (1024, 1024, 3)
(0, 151, 1024, 873)
0.17486338797814208
Image molded
Meta of image prepared
(1, 1024, 1024, 3)
anchors shape is (1, 261888, 4) float32
Tensor("input_image:0", shape=(?, ?, ?, 3), dtype=float32, device=/device:CPU:0)
Tensor("input_anchors:0", shape=(?, ?, 4), dtype=float32, device=/device:CPU:0)
Tensor("input_image_meta:0", shape=(?, 14), dtype=float32, device=/device:CPU:0)
Found  Tensor("output_detections:0", shape=(2, 100, 6), dtype=float32)
Found  Tensor("output_mrcnn_class:0", shape=(?, 1000, 2), dtype=float32)
Found  Tensor("output_mrcnn_bbox:0", shape=(?, 1000, 2, 4), dtype=float32)
Found  Tensor("output_mrcnn_mask:0", shape=(2, 100, 28, 28, 2), dtype=float32)
Found  Tensor("output_rois:0", shape=(2, ?, ?), dtype=float32)
Windows (1, 4)   [[   0  151 1024  873]]
Traceback (most recent call last):
  File "infere_from_pb.py", line 318, in <module>
    tf.app.run()
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "infere_from_pb.py", line 78, in main
    img_anchors_ph: image_anchors})
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
    run_metadata_ptr)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run
    str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (1, 93) for Tensor 'input_image_meta:0', which has shape '(?, 14)'

Anyone knows why I got this?

I solved that. I just have 2 classes. I had to change to inference_config.NUM_CLASSES = 2 from the script of @parai . Now I am getting a new error after running this line:


detections = sess.run(detectionsT, feed_dict={img_ph: molded_images, img_meta_ph: image_metas,
                                                      img_anchors_ph: image_anchors})

Error:

2018-12-20 15:05:52.008913: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at strided_slice_op.cc:106 : Invalid argument: slice index 1 of dimension 0 out of bounds.
2018-12-20 15:05:52.009481: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at strided_slice_op.cc:106 : Invalid argument: slice index 1 of dimension 0 out of bounds.
Traceback (most recent call last):
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
    return fn(*args)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: slice index 1 of dimension 0 out of bounds.
     [[{{node mrcnn_detection/strided_slice_34}} = StridedSlice[Index=DT_INT32, T=DT_FLOAT, begin_mask=0, ellipsis_mask=0, end_mask=0, new_axis_mask=0, shrink_axis_mask=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](mrcnn_detection/truediv, mrcnn_detection/strided_slice_8/stack_1, mrcnn_detection/strided_slice_31/stack_1, mrcnn_detection/strided_slice_8/stack_1)]]
     [[{{node res5c_branch2c/bias/_426}} = _Send[T=DT_FLOAT, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_1078_res5c_branch2c/bias", _device="/job:localhost/replica:0/task:0/device:GPU:0"](res5c_branch2c/bias)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/javier/.pycharm_helpers/pydev/pydevd.py", line 1664, in <module>
    main()
  File "/home/javier/.pycharm_helpers/pydev/pydevd.py", line 1658, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/home/javier/.pycharm_helpers/pydev/pydevd.py", line 1068, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/home/javier/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/javier/disk_user/OCR_project/Mask_RCNN/infering/infere_from_pb.py", line 318, in <module>
    tf.app.run()
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "/home/javier/disk_user/OCR_project/Mask_RCNN/infering/infere_from_pb.py", line 78, in main
    img_anchors_ph: image_anchors})
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
    run_metadata_ptr)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
    run_metadata)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: slice index 1 of dimension 0 out of bounds.
     [[node mrcnn_detection/strided_slice_34 (defined at /home/javier/disk_user/OCR_project/Mask_RCNN/infering/infere_from_pb.py:29)  = StridedSlice[Index=DT_INT32, T=DT_FLOAT, begin_mask=0, ellipsis_mask=0, end_mask=0, new_axis_mask=0, shrink_axis_mask=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](mrcnn_detection/truediv, mrcnn_detection/strided_slice_8/stack_1, mrcnn_detection/strided_slice_31/stack_1, mrcnn_detection/strided_slice_8/stack_1)]]
     [[{{node res5c_branch2c/bias/_426}} = _Send[T=DT_FLOAT, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_1078_res5c_branch2c/bias", _device="/job:localhost/replica:0/task:0/device:GPU:0"](res5c_branch2c/bias)]]

Caused by op 'mrcnn_detection/strided_slice_34', defined at:
  File "/home/javier/.pycharm_helpers/pydev/pydevd.py", line 1664, in <module>
    main()
  File "/home/javier/.pycharm_helpers/pydev/pydevd.py", line 1658, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/home/javier/.pycharm_helpers/pydev/pydevd.py", line 1068, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/home/javier/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/javier/disk_user/OCR_project/Mask_RCNN/infering/infere_from_pb.py", line 318, in <module>
    tf.app.run()
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "/home/javier/disk_user/OCR_project/Mask_RCNN/infering/infere_from_pb.py", line 29, in main
    _ = tf.import_graph_def(graph_def, name='')
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 442, in import_graph_def
    _ProcessNewOps(graph)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 234, in _ProcessNewOps
    for new_op in graph._add_new_tf_operations(compute_devices=False):  # pylint: disable=protected-access
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3440, in _add_new_tf_operations
    for c_op in c_api_util.new_tf_operations(self)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3440, in <listcomp>
    for c_op in c_api_util.new_tf_operations(self)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3299, in _create_op_from_tf_operation
    ret = Operation(c_op, self)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1770, in __init__
    self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): slice index 1 of dimension 0 out of bounds.
     [[node mrcnn_detection/strided_slice_34 (defined at /home/javier/disk_user/OCR_project/Mask_RCNN/infering/infere_from_pb.py:29)  = StridedSlice[Index=DT_INT32, T=DT_FLOAT, begin_mask=0, ellipsis_mask=0, end_mask=0, new_axis_mask=0, shrink_axis_mask=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](mrcnn_detection/truediv, mrcnn_detection/strided_slice_8/stack_1, mrcnn_detection/strided_slice_31/stack_1, mrcnn_detection/strided_slice_8/stack_1)]]
     [[{{node res5c_branch2c/bias/_426}} = _Send[T=DT_FLOAT, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_1078_res5c_branch2c/bias", _device="/job:localhost/replica:0/task:0/device:GPU:0"](res5c_branch2c/bias)]]

Anyone knows what would happen?

ashnair1 commented 5 years ago

@parai I exported the model and then tried using the infere_from_pb.py script to run inference but ended up with this error: KeyError: "The name 'input_image_1:0' refers to a Tensor which does not exist. The operation, 'input_image_1', does not exist in the graph."

Edit: Fixed it. My problem was a naming issue. Instead of input_image_1:0 it should have been input_image:0

Using @fastlater 's implementation I'm getting this error when running inference: Cannot feed value of shape (1, 10) for Tensor 'input_image_meta:0', which has shape '(?, 14)'. This is the case using the infere_from_pb.py script and the notebook from @fastlater. I only have 2 classes (Background and building). it seems that some people have encountered this issue previously like @kuonangzhe and @javierfs , if so could you kindly tell me how you fixed it?

Edit: Got around it by reshaping the image_meta array.

On a side note has anyone been able to successfully go through the entire pipeline of exporting the model and using it for inference? There are so many variations here of exporters and importers which is a bit confusing.

ashnair1 commented 5 years ago

So I was able to go through the entire pipeline of exporting and importing the model for inference. However the model isn't able to do inference correctly. I tried out the same model before and after exporting it to a .pb file on the same image to compare their results.

Standard Inference model ROIs:
[[322 711 386 786] [384 728 597 886] [700 379 790 442] [718 0 747 34] [290 256 441 635] [754 224 802 249] [ 22 367 41 403] [ 29 4 208 218] [378 161 445 270] [697 757 732 790] [768 512 799 543] [848 759 887 787] [ 17 324 39 352] [769 631 811 651] [731 629 823 692] [254 746 292 784] [886 124 900 204] [698 470 751 517] [765 475 791 538] [326 167 451 369] [694 742 788 797] [ 0 42 39 86] [657 130 683 142]]

Imported .pb model: ROIs: [[1473 711 1474 786] [1473 712 1474 783] [1473 379 1474 442] [1473 382 1474 443] [1473 165 1474 276] [1473 0 1474 34] [1473 224 1474 249] [1473 367 1474 403] [1473 161 1474 270] [1473 223 1474 247] [1473 365 1474 400] [1473 223 1474 247] [1473 168 1474 281] [1473 757 1474 790] [1473 760 1474 792] [1473 157 1474 285] [1473 317 1474 649] [1473 512 1474 543] [1473 364 1474 399] [1473 759 1474 787] [1473 379 1474 449] [1473 202 1474 669] [1473 324 1474 352] [1473 305 1474 682] [1473 222 1474 247] [1473 161 1474 278] [1473 0 1474 34] [1473 379 1474 440] [1473 631 1474 651] [1473 222 1474 248] [1473 379 1474 445] [1473 152 1474 399] [1473 629 1474 692] [1473 365 1474 400] [1473 746 1474 784] [1473 124 1474 204] [1473 127 1474 210] [1473 322 1474 351] [1473 403 1474 447] [1473 223 1474 249] [1473 366 1474 401] [1473 160 1474 312] [1473 470 1474 517] [1473 128 1474 213] [1473 475 1474 538] [1473 158 1474 262] [1473 224 1474 251] [1473 474 1474 513] [1473 629 1474 648] [1473 742 1474 797] [1473 42 1474 86] [1473 323 1474 350] [1473 513 1474 544] [1473 390 1474 448] [1473 470 1474 520] [1473 402 1474 447] [1473 473 1474 514] [1473 130 1474 142] [1473 386 1474 445] [1473 381 1474 444]]

So the model is,

Detecting more objects than are actually present
The first and third elements of the bounding box (x coordinate and width I believe) are stuck at a particular value (1473, 1474). This is also seen while testing on other images as well.

Any hints as to what could be the reason?

Edit: Never mind. Fixed it by rewriting the importer and exporter. Works fine now.

a-ershov commented 5 years ago

I received the .pb file, but I cannot use it for TensorFlow serving. 2019-02-12 12:39:17.963412: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve } 2019-02-12 12:39:17.975561: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:259] SavedModel load for tags { serve }; Status: fail. Took 99931 microseconds. 2019-02-12 12:39:17.975713: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: r version: 1} failed: Not found: Could not find meta graph def matching supplied tags: { serve }. To inspect available tag-sets in the SavedModel, please use the SavedModel CLI: 'saved_model_cli'

And when I try to check using saved_model_cli I got error RuntimeError: MetaGraphDef associated with tag-set could not be found in SavedModel

How to use the model with TF Serving?

eoghanmull commented 5 years ago

@chohaku84 New script here

What is the config file for --config_file=X ?

I had no config file for my training but set configurations through the config class.

benDavidson6 commented 5 years ago

This will create a frozen graph which you can use

import keras.backend as K
import tensorflow as tf

# I needed to add this
sess = tf.Session()
K.set_session(sess)

from mrcnn import model as modellib
# my config subclass
from network_configs import ExampleConfig

def freeze_session(session, keep_var_names=None, output_names=None, clear_devices=True):
    """
    Freezes the state of a session into a pruned computation graph.

    Creates a new computation graph where variable nodes are replaced by
    constants taking their current value in the session. The new graph will be
    pruned so subgraphs that are not necessary to compute the requested
    outputs are removed.
    @param session The TensorFlow session to be frozen.
    @param keep_var_names A list of variable names that should not be frozen,
                          or None to freeze all the variables in the graph.
    @param output_names Names of the relevant graph outputs.
    @param clear_devices Remove the device directives from the graph for better portability.
    @return The frozen graph definition.
    """
    graph = sess.graph

    with graph.as_default():
        freeze_var_names = list(set(v.op.name for v in tf.global_variables()).difference(keep_var_names or []))

        output_names = output_names or []
        input_graph_def = graph.as_graph_def()

        if clear_devices:
            for node in input_graph_def.node:
                node.device = ""

        frozen_graph = tf.graph_util.convert_variables_to_constants(
            session, input_graph_def, output_names, freeze_var_names)
        return frozen_graph

def freeze_model(model, name):
    frozen_graph = freeze_session(
        sess,
        output_names=[out.op.name for out in model.outputs][:4])
    directory = '.\\'
    tf.train.write_graph(frozen_graph, directory, name + '.pb', as_text=False)

config = ExampleConfig()
MODEL_DIR = 'where your weights live'
H5_WEIGHT_PATH = 'path to h5 file'
FROZEN_NAME = 'frozen_graph.pb
model = modellib.MaskRCNN(
    mode="inference",
    config=config,
    model_dir=MODEL_DIR)
model.load_weights(H5_WEIGHT_PATH, by_name=True)
freeze_model(model.keras_model, FROZEN_NAME)

pknambiar commented 5 years ago

@benDavidson6 Hello, Thanks for your code snippet. But, I didn't quite understand the first variable amongst these two below:

MODEL_DIR = 'where your weights live' H5_WEIGHT_PATH = 'path to h5 file'

Is it that the former is the (input) PATH/DIR and the latter is the (input) .h5 file name ?

benDavidson6 commented 5 years ago

ile'

Is it that the former is the (input)

The model dir is the folder your weights are in, and the path is the actual path to the weights.

pknambiar commented 5 years ago

Hi, The below line is not getting resolved in my code: from network_configs import ExampleConfig Where do I find network_configs ?

benDavidson6 commented 5 years ago

That is my code. You need to adapt it so that you have a config object like config = ExampleConfig(). You must have created a config object somewhere to run any training or inference.

davidb1 commented 5 years ago

Anyone knows how to get past the "Unknown layer: BatchNorm" error?

marcosbis commented 5 years ago

Anyone knows how to get past the "Unknown layer: BatchNorm" error?

Im having the same problem

benDavidson6 commented 5 years ago

forward.zip

In the attached code there is a class Forward which you just give your config, and path to the pb file then you can do:

forward_model = Forward('path/to/pb/', my_config)
results = forward_model(list_of_image_arrays)

changya1990 commented 5 years ago

forward.zip

In the attached code there is a class Forward which you just give your config, and path to the pb file then you can do:
forward_model = Forward('path/to/pb/', my_config)
results = forward_model(list_of_image_arrays)

Thanks, the result is right when the input node "input_image:0, input_image_meta:0" of my inference code changed to ''import/input_image:0, import/input_image_meta:0".

navinthenapster commented 5 years ago

@Surmeh Code here

This link is not available. Could you provide other link

adebiasio-estilos commented 5 years ago

How to use the model with TF Serving?

I have also this problem @Vellar , did you find any solution?

bendangnuksung commented 5 years ago

I converted the keras model to tensorflow frozen pb and tensorflow saved model. Code here.

farmaker47 commented 5 years ago

This worked for me. Pay attention to this line: keras_to_tensorflow(model.keras_model,output_dir=output_dir,model_name="keras_segment.pb") "where you have to write .....model.keras_model"

import os
import tensorflow as tf
import keras.backend as K

"Method for saving keras to tensorflow to a specific directory"
def keras_to_tensorflow(keras_model, output_dir, model_name,out_prefix="output_", 
log_tensorboard=True):

if os.path.exists(output_dir) == False:
    os.mkdir(output_dir)

out_nodes = []

for i in range(len(keras_model.outputs)):
    out_nodes.append(out_prefix + str(i + 1))
    tf.identity(keras_model.output[i], out_prefix + str(i + 1))

sess = K.get_session()

from tensorflow.python.framework import graph_util, graph_io

init_graph = sess.graph.as_graph_def()

main_graph = tf.compat.v1.graph_util.convert_variables_to_constants(sess, init_graph, out_nodes)

graph_io.write_graph(main_graph, output_dir, name=model_name, as_text=False)

if log_tensorboard:
    from tensorflow.python.tools import import_pb_to_tensorboard

    import_pb_to_tensorboard.import_to_tensorboard(
        os.path.join(output_dir, model_name),
        output_dir)

"Load weights"
 model.load_weights('mask_rcnn_coco.h5', by_name=True)

"Make directory"
output_dir = os.path.join(os.getcwd(),"checkpoint")

"Saving to Tensorflow"
keras_to_tensorflow(model.keras_model,output_dir=output_dir,model_name="keras_segment.pb")

print("Success!")

frk1993 commented 5 years ago

I have the following issue:

this is root_dir: /home/javier/disk_user/OCR_project/Mask_RCNN/infering
this is Image_dir: /home/javier/disk_user/OCR_project/Mask_RCNN/infering/../customImages/val/
choose image file is 490_2.jpg
(5856, 4128, 3)
Processing 1 images
image                    shape: (5856, 4128, 3)       min:    0.00000  max:  255.00000  uint8
RGB image loaded and preprocessed.
/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/skimage/transform/_warps.py:110: UserWarning: Anti-aliasing will be enabled by default in skimage 0.15 to avoid aliasing artifacts when down-sampling images.
  warn("Anti-aliasing will be enabled by default in skimage 0.15 to "
(5856, 4128, 3)
Image resized at:  (1024, 1024, 3)
(0, 151, 1024, 873)
0.17486338797814208
Image molded
Meta of image prepared
(1, 1024, 1024, 3)
anchors shape is (1, 261888, 4) float32
Tensor("input_image:0", shape=(?, ?, ?, 3), dtype=float32, device=/device:CPU:0)
Tensor("input_anchors:0", shape=(?, ?, 4), dtype=float32, device=/device:CPU:0)
Tensor("input_image_meta:0", shape=(?, 14), dtype=float32, device=/device:CPU:0)
Found  Tensor("output_detections:0", shape=(2, 100, 6), dtype=float32)
Found  Tensor("output_mrcnn_class:0", shape=(?, 1000, 2), dtype=float32)
Found  Tensor("output_mrcnn_bbox:0", shape=(?, 1000, 2, 4), dtype=float32)
Found  Tensor("output_mrcnn_mask:0", shape=(2, 100, 28, 28, 2), dtype=float32)
Found  Tensor("output_rois:0", shape=(2, ?, ?), dtype=float32)
Windows (1, 4)   [[   0  151 1024  873]]
Traceback (most recent call last):
  File "infere_from_pb.py", line 318, in <module>
    tf.app.run()
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "infere_from_pb.py", line 78, in main
    img_anchors_ph: image_anchors})
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
    run_metadata_ptr)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run
    str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (1, 93) for Tensor 'input_image_meta:0', which has shape '(?, 14)'

Anyone knows why I got this?

I solved that. I just have 2 classes. I had to change to inference_config.NUM_CLASSES = 2 from the script of @parai . Now I am getting a new error after running this line:


detections = sess.run(detectionsT, feed_dict={img_ph: molded_images, img_meta_ph: image_metas,
                                                      img_anchors_ph: image_anchors})

Error:

2018-12-20 15:05:52.008913: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at strided_slice_op.cc:106 : Invalid argument: slice index 1 of dimension 0 out of bounds.
2018-12-20 15:05:52.009481: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at strided_slice_op.cc:106 : Invalid argument: slice index 1 of dimension 0 out of bounds.
Traceback (most recent call last):
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
    return fn(*args)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: slice index 1 of dimension 0 out of bounds.
   [[{{node mrcnn_detection/strided_slice_34}} = StridedSlice[Index=DT_INT32, T=DT_FLOAT, begin_mask=0, ellipsis_mask=0, end_mask=0, new_axis_mask=0, shrink_axis_mask=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](mrcnn_detection/truediv, mrcnn_detection/strided_slice_8/stack_1, mrcnn_detection/strided_slice_31/stack_1, mrcnn_detection/strided_slice_8/stack_1)]]
   [[{{node res5c_branch2c/bias/_426}} = _Send[T=DT_FLOAT, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_1078_res5c_branch2c/bias", _device="/job:localhost/replica:0/task:0/device:GPU:0"](res5c_branch2c/bias)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/javier/.pycharm_helpers/pydev/pydevd.py", line 1664, in <module>
    main()
  File "/home/javier/.pycharm_helpers/pydev/pydevd.py", line 1658, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/home/javier/.pycharm_helpers/pydev/pydevd.py", line 1068, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/home/javier/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/javier/disk_user/OCR_project/Mask_RCNN/infering/infere_from_pb.py", line 318, in <module>
    tf.app.run()
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "/home/javier/disk_user/OCR_project/Mask_RCNN/infering/infere_from_pb.py", line 78, in main
    img_anchors_ph: image_anchors})
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
    run_metadata_ptr)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
    run_metadata)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: slice index 1 of dimension 0 out of bounds.
   [[node mrcnn_detection/strided_slice_34 (defined at /home/javier/disk_user/OCR_project/Mask_RCNN/infering/infere_from_pb.py:29)  = StridedSlice[Index=DT_INT32, T=DT_FLOAT, begin_mask=0, ellipsis_mask=0, end_mask=0, new_axis_mask=0, shrink_axis_mask=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](mrcnn_detection/truediv, mrcnn_detection/strided_slice_8/stack_1, mrcnn_detection/strided_slice_31/stack_1, mrcnn_detection/strided_slice_8/stack_1)]]
   [[{{node res5c_branch2c/bias/_426}} = _Send[T=DT_FLOAT, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_1078_res5c_branch2c/bias", _device="/job:localhost/replica:0/task:0/device:GPU:0"](res5c_branch2c/bias)]]

Caused by op 'mrcnn_detection/strided_slice_34', defined at:
  File "/home/javier/.pycharm_helpers/pydev/pydevd.py", line 1664, in <module>
    main()
  File "/home/javier/.pycharm_helpers/pydev/pydevd.py", line 1658, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/home/javier/.pycharm_helpers/pydev/pydevd.py", line 1068, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/home/javier/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/javier/disk_user/OCR_project/Mask_RCNN/infering/infere_from_pb.py", line 318, in <module>
    tf.app.run()
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "/home/javier/disk_user/OCR_project/Mask_RCNN/infering/infere_from_pb.py", line 29, in main
    _ = tf.import_graph_def(graph_def, name='')
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 442, in import_graph_def
    _ProcessNewOps(graph)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 234, in _ProcessNewOps
    for new_op in graph._add_new_tf_operations(compute_devices=False):  # pylint: disable=protected-access
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3440, in _add_new_tf_operations
    for c_op in c_api_util.new_tf_operations(self)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3440, in <listcomp>
    for c_op in c_api_util.new_tf_operations(self)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3299, in _create_op_from_tf_operation
    ret = Operation(c_op, self)
  File "/home/javier/anaconda3/envs/AIenv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1770, in __init__
    self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): slice index 1 of dimension 0 out of bounds.
   [[node mrcnn_detection/strided_slice_34 (defined at /home/javier/disk_user/OCR_project/Mask_RCNN/infering/infere_from_pb.py:29)  = StridedSlice[Index=DT_INT32, T=DT_FLOAT, begin_mask=0, ellipsis_mask=0, end_mask=0, new_axis_mask=0, shrink_axis_mask=1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](mrcnn_detection/truediv, mrcnn_detection/strided_slice_8/stack_1, mrcnn_detection/strided_slice_31/stack_1, mrcnn_detection/strided_slice_8/stack_1)]]
   [[{{node res5c_branch2c/bias/_426}} = _Send[T=DT_FLOAT, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_1078_res5c_branch2c/bias", _device="/job:localhost/replica:0/task:0/device:GPU:0"](res5c_branch2c/bias)]]

Anyone knows what would happen?

Hi, I am facing the same error. Could you solve this problem ?

best regards

buaacarzp commented 5 years ago

@Surmeh Code here

I try to open the web from you give ,but it give me a 404 error ....

buaacarzp commented 5 years ago

forward.zip In the attached code there is a class Forward which you just give your config, and path to the pb file then you can do:
forward_model = Forward('path/to/pb/', my_config)
results = forward_model(list_of_image_arrays)
Thanks, the result is right when the input node "input_image:0, input_image_meta:0" of my inference code changed to ''import/input_image:0, import/input_image_meta:0".

hey, I am the chinese too, my qq num is 510695983, I meet the same trouble with you ,but I didn't solve it , can you help me ?

angyee commented 5 years ago

@Surmeh Code here

please share code.. i have only .h5 then how to convert into .pb

Surmeh commented 5 years ago

Hi @angyee, the code was shared by @ericj974.

angyee commented 5 years ago

@jmtatsch To produce h5: python3 coco.py evaluate --dataset=$COCO_PATH --model=coco

To save model in coco.py: evaluate_coco(model, dataset_val, coco, "bbox", limit=int(args.limit)) model.keras_model.save("mrcnn_eval.h5")

Extracting pb from h5: python3 keras_to_tensorflow.py -input_model_file saved_model_mrcnn_eval.h5 -output_model_file model.pb -num_outputs=7

Could you paste the full stacktrace?

i am running

sudo python3 keras_to_tensorflow.py --input_model=/home/deepedge/mask_rcnn-master/mask_rcnn_damage_0010.h5 --output_model=/home/deepedge/mask_rcnn-master/dent.pb

and getting

Using TensorFlow backend. E0619 15:22:44.379571 140399717234496 keras_to_tensorflow.py:95] Input file specified only holds the weights, and not the model definition. Save the model using model.save(filename.h5) which will contain the network architecture as well as its weights. If the model is saved using the model.save_weights(filename) function, either input_model_json or input_model_yaml flags should be set to to import the network architecture prior to loading the weights. Check the keras documentation for more details (https://keras.io/getting-started/faq/) Traceback (most recent call last): File "keras_to_tensorflow.py", line 182, in app.run(main) File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 300, in run _run_main(main, args) File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 251, in _run_main sys.exit(main(argv)) File "keras_to_tensorflow.py", line 128, in main model = load_model(FLAGS.input_model, FLAGS.input_model_json, FLAGS.input_model_yaml) File "keras_to_tensorflow.py", line 106, in load_model raise wrong_file_err File "keras_to_tensorflow.py", line 63, in load_model model = keras.models.load_model(input_model_path) File "/usr/local/lib/python3.6/dist-packages/keras/engine/saving.py", line 419, in load_model model = _deserialize_model(f, custom_objects, compile) File "/usr/local/lib/python3.6/dist-packages/keras/engine/saving.py", line 221, in _deserialize_model model_config = f['model_config'] File "/usr/local/lib/python3.6/dist-packages/keras/utils/io_utils.py", line 302, in getitem raise ValueError('Cannot create group in read only mode.') ValueError: Cannot create group in read only mode.

@Cpruce @Surmeh @waleedka @PavlosMelissinos

ashish9277 commented 5 years ago

I am getting the same error as above. The error is occurring with original weights as well as with the custom Mask RCNN weights.

lukeb94 commented 5 years ago

Could someone please inform me of the steps required to output a .h5 model file (not weights) that can then be converted to .pb. I understand how to convert to .pb but struggling with generating the model file. The above comments are somewhat helpful but some links are broken.

Help would be greatly appreciated! I have successfully setup a custom object detector with great results.

angyee commented 5 years ago

You can download the forward.zip provided in the comment and use the code below to inference it.

from forward import ForwardModel import cv2 import numpy as np

frozen_graph_path = "frozen_model/mask_frozen_graph.pb"

assign your own Config class

my_config = YOUR_CONFIG_CLASS

forward_model = ForwardModel(frozen_graph_path, my_config)

def test_one_image(image): images = np.expand_dims(image, axis=0) results = forward_model(images) return results

if name == "main": test_image_path = 'XYZ.jpg' image = cv2.imread(test_image_path) r = test_one_image(image) print(r)

how to set my_config()?

@bendangnuksung @ashish9277 @gakarak

lukeb94 commented 5 years ago

But how do we obtain either the .pb model file or the .h5 model file (not weight file) to then be converted into .pb? I am deploying the model on mobile using server.

lukeb94 commented 5 years ago

Hi all, I successfully generated a .pb file with the help of #1126. Could someone please let me know how I can run inference on this .pb model file to test its functionality? Many thanks

angyee commented 5 years ago

Hi all, I successfully generated a .pb file with the help of #1126. Could someone please let me know how I can run inference on this .pb model file to test its functionality? Many thanks

I have also created the .pb by using now how to generate and display the masked image. https://github.com/bendangnuksung/mrcnn_serving_ready

please solve same problem here please run the inference and share here https://github.com/bendangnuksung/mrcnn_serving_ready/issues/5

@lukeb94 @bendangnuksung

lukeb94 commented 5 years ago

Hi all, I successfully generated a .pb file with the help of #1126. Could someone please let me know how I can run inference on this .pb model file to test its functionality? Many thanks

I have also created the .pb by using now how to generate and display the masked image. https://github.com/bendangnuksung/mrcnn_serving_ready

please solve same problem here please run the inference and share here bendangnuksung/mrcnn_serving_ready#5

@lukeb94 @bendangnuksung

Yes, I managed to generate the .pb file. But how can we now test detentions?

Does this help? Very keen to get it working. https://github.com/parai/Mask_RCNN/commit/6289c1bd08fc90a1c3e296be8155674651f82a4b

AI-ML-Enthusiast commented 5 years ago

@ericj974 @ps48 I created .pb file but how can I create .pbtxt file ot use in opencv

AI-ML-Enthusiast commented 5 years ago

hello all, what about .pbtxt file? would help me please. I created .pb but not .pbtxt. I need those files for opencv dnn @ps48 @ericj974 @Surmeh @luoshanwei @lukeb94

angyee commented 5 years ago

hello all, what about .pbtxt file? would help me please. I created .pb but not .pbtxt. I need those files for opencv dnn @ps48 @ericj974 @Surmeh @luoshanwei @lukeb94

https://gist.github.com/Arafatk/c063bddb9b8d17a037695d748db4f592

convert .pb to pbtxt after that how to Display result please share me also

matterport / Mask_RCNN

How to convert Mask rcnn model to Tensorflow .pb #218

assign your own Config class