search

patlevin / tfjs-to-tf

A TensorFlow.js Graph Model Converter
MIT License
139 stars 18 forks source link

SavedModel doesn't have SignatureDefs #13

Closed glenvorel closed 4 years ago

glenvorel commented 4 years ago

Conversion of TFJS model (PoseNet ResNet50) to SavedModel is successful but the produced SavedModel doesn't seem to contain any signature keys, inputs and outputs.

saved_model_cli show --dir /posenet/savedmodel --all
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

# end out output

It means that TensorFlow Serving cannot serve this model.

My idea was to load the produced model as GraphDef

def get_graph_def_from_saved_model(saved_model_dir):
    with tf.Session() as session:
        meta_graph_def = tf.saved_model.loader.load(
            session,
            tags=['serve'],
            export_dir=saved_model_dir
        )
    return meta_graph_def.graph_def
graph_def = get_graph_def_from_saved_model('/posenet/savedmodel')

find name of the input

input_nodes = ([node.name for node in graph_def.node if node.op == 'Placeholder'])
print(input_nodes)  # ['sub_2']

and then save it again

with tf.Session(graph=tf.Graph()) as session:
    tf.import_graph_def(graph_def, name='')
    inputs = {input_name: session.graph.get_tensor_by_name(f'{input_name}:0') for input_name in input_nodes}
    outputs =  # how do I find the outputs?
    tf.saved_model.simple_save(
        session,
        '/posenet/savedmodel_2',
        inputs=inputs,
        outputs=outputs
    )

but I don't know how to find names of the output nodes. Can you please suggest? Or can you think of a better way of generating SignatureDefs for the SavedModel?

Thank you for this useful tool!

patlevin commented 4 years ago

I consider this issue to be a bug, I will work on an update to fix this.

Thanks for pointing this out!

floe commented 4 years ago

Seconded. Exact same issue when converting body-pix, resulting saved_model does not have any SignatureDefs.

glenvorel commented 4 years ago

Workaround before the fix is available.

  1. Convert the model generated by TensorFlow.js Graph Model Converter to a graph that can be inspected in TensorBoard.

    1. When the format is SavedModel, use import_pb_to_tensorboard.py from the master branch. 
      python import_pb_to_tensorboard.py --model_dir /posenet/saved_model --log_dir /posenet/log_dir
    2. When the format is frozen model, use import_pb_to_tensorboard.py from the r2.1 branch - it was the last time the tool supported frozen models. 
      python import_pb_to_tensorboard.py --model_dir /posenet/frozen_model.pb --log_dir /posenet/log_dir

  2. Load the graph in TensorBoard.

    tensorboard --logdir=/posenet/log_dir

  3. Open TensorBoard (localhost:6006) and identify the input and output nodes. In the case of PoseNet, there is 1 input (sub_2) are 2 outputs (float_heatmaps and float_short_offsets). tensorboard_posenet

  4. Save the SavedModel with SignatureDefs. (Note: This works in TF1.x; when using TF2.x, some imports must be done from tf.compat.v1)

    def get_graph_def_from_saved_model(saved_model_dir):
    with tf.Session() as session:
        meta_graph_def = tf.saved_model.loader.load(
            session,
            tags=['serve'],
            export_dir=saved_model_dir
        )
    return meta_graph_def.graph_def

graph_def = get_graph_def_from_saved_model('/posenet/saved_model')

input_nodes = ['sub_2']
output_nodes = ['float_heatmaps', 'float_short_offsets']

with tf.Session(graph=tf.Graph()) as session:
    tf.import_graph_def(graph_def, name='')
    inputs = {input_node: session.graph.get_tensor_by_name(f'{input_node}:0') for input_node in input_nodes}
    outputs = {output_node: session.graph.get_tensor_by_name(f'{output_node}:0') for output_node in output_nodes}
    tf.saved_model.simple_save(
        session,
        '/posenet/savedmodel_signaturedefs',
        inputs=inputs,
        outputs=outputs
    )

  5. Done! Inspect the new SavedModel, it should now contain SignatureDefs.

    saved_model_cli show --dir /posenet/savedmodel_signaturedefs --all

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['sub_2'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, -1, -1, 3)
        name: sub_2:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['float_heatmaps'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, -1, -1, 17)
        name: float_heatmaps:0
    outputs['float_short_offsets'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, -1, -1, 34)
        name: float_short_offsets:0
  Method name is: tensorflow/serving/predict
floe commented 4 years ago

@glenvorel thanks for the workaround! Also works with TF2 if you use tf.compat.v1 prefix for everything. JFYI, for steps 1-3, you can also use Netron instead (https://lutzroeder.github.io/netron/).

floe commented 4 years ago

Additional comment: if you want to continue to convert the result to TFLite (which I did), then the following code snippet from the converter API guide may be helpful (this can basically be appended right below the code example from @glenvorel ):

model = tf.saved_model.load(export_dir)
concrete_func = model.signatures[
  tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY]
concrete_func.inputs[0].set_shape([1, 256, 256, 3])
converter = TFLiteConverter.from_concrete_functions([concrete_func])
glenvorel commented 4 years ago

Thank you for the quick fix! While testing it, I ran into some import issues.

The traceback below is common when I:

Traceback (most recent call last):
  File "/usr/local/bin/tfjs_graph_converter", line 5, in <module>
    from tfjs_graph_converter.converter import pip_main
  File "/usr/local/lib/python3.6/dist-packages/tfjs_graph_converter/__init__.py", line 4, in <module>
    from tfjs_graph_converter import api                    # noqa: F401
  File "/usr/local/lib/python3.6/dist-packages/tfjs_graph_converter/api.py", line 29, in <module>
    from tfjs_graph_converter.optimization import optimize_graph
  File "/usr/local/lib/python3.6/dist-packages/tfjs_graph_converter/optimization.py", line 19, in <module>
    from tfjs_graph_converter.util import get_input_nodes, get_output_nodes
  File "/usr/local/lib/python3.6/dist-packages/tfjs_graph_converter/util.py", line 13, in <module>
    from tensorflow_core.core.protobuf.meta_graph_pb2 import SignatureDef
ModuleNotFoundError: No module named 'tensorflow_core'

This comment says that tensorflow_core only exists in 1.15, 2.0 and 2.1 so I gave it another try:

The tensorflow_core module was imported successfully but there was an issue importing AttrValue.

Traceback (most recent call last):
  File "/usr/local/bin/tfjs_graph_converter", line 5, in <module>
    from tfjs_graph_converter.converter import pip_main
  File "/usr/local/lib/python3.6/dist-packages/tfjs_graph_converter/__init__.py", line 4, in <module>
    from tfjs_graph_converter import api                    # noqa: F401
  File "/usr/local/lib/python3.6/dist-packages/tfjs_graph_converter/api.py", line 29, in <module>
    from tfjs_graph_converter.optimization import optimize_graph
  File "/usr/local/lib/python3.6/dist-packages/tfjs_graph_converter/optimization.py", line 19, in <module>
    from tfjs_graph_converter.util import get_input_nodes, get_output_nodes
  File "/usr/local/lib/python3.6/dist-packages/tfjs_graph_converter/util.py", line 13, in <module>
    from tensorflow_core.core.protobuf.meta_graph_pb2 import SignatureDef
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/__init__.py", line 46, in <module>
    from . _api.v2 import compat
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/_api/v2/compat/__init__.py", line 39, in <module>
    from . import v1
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/_api/v2/compat/v1/__init__.py", line 32, in <module>
    from . import compat
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/_api/v2/compat/v1/compat/__init__.py", line 39, in <module>
    from . import v1
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/_api/v2/compat/v1/compat/v1/__init__.py", line 82, in <module>
    from tensorflow.python import AttrValue
ImportError: cannot import name 'AttrValue'
  1. I suggest that if TensorFlow.js Graph Model Converter depends on TF==2.1.0 and doesn't support TF>2.1.0, the documentation should mention it.
  2. I would like to ask if there is some other dependency needed to solve the AttrValue import error? Below is pip freeze of tensorflow/tensorflow:2.1.0-py3 and pip install tfjs-graph-converter==1.1.0:
absl-py==0.9.0
asn1crypto==0.24.0
astor==0.8.1
astunparse==1.6.3
cachetools==4.0.0
certifi==2019.11.28
chardet==3.0.4
cryptography==2.1.4
gast==0.3.3
google-auth==1.10.0
google-auth-oauthlib==0.4.1
google-pasta==0.1.8
grpcio==1.26.0
h5py==2.10.0
idna==2.6
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.2
keyring==10.6.0
keyrings.alt==3.0
Markdown==3.1.1
numpy==1.18.1
oauthlib==3.1.0
opt-einsum==3.1.0
prompt-toolkit==1.0.14
protobuf==3.11.2
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycrypto==2.6.1
Pygments==2.6.1
pygobject==3.26.1
PyInquirer==1.0.3
pyxdg==0.25
regex==2020.7.14
requests==2.22.0
requests-oauthlib==1.3.0
rsa==4.0
scipy==1.4.1
SecretStorage==2.3.1
six==1.13.0
tensorboard==2.3.0
tensorboard-plugin-wit==1.7.0
tensorflow==2.1.0
tensorflow-cpu==2.3.0
tensorflow-estimator==2.3.0
tensorflow-hub==0.7.0
tensorflowjs==2.0.1.post1
termcolor==1.1.0
tfjs-graph-converter==1.1.0
urllib3==1.25.7
wcwidth==0.2.5
Werkzeug==0.16.0
wrapt==1.11.2
patlevin commented 4 years ago

@glenvorel Thank you for making me aware of the compatibility issue! Your tensorflow install seems to be broken, though. The versions of tensorflow-cpu and tensorflow don't seem to match. You basically have a mixture of TF 2.1 and TF 2.3 installed and that could be the issue here.

I will do some version-compatibility testing with TF 2.2 and TF 2.3 and release an update late today. In the meantime make sure that all tensorflow-* packages in your environment have matching version numbers to avoid problems. The AttrValue-issue stems from within tensorflow itself, likely due to the aforementioned version conflict.

patlevin commented 4 years ago

@glenvorel The package now works as-is with tensorflow v2.2 and v2.3 as well. You can just use an empty (conda-)environment and use pip install tfjs_graph_converter, which will pull in all dependencies in the latest versions (including TF v2.3) and works as-is.

glenvorel commented 4 years ago

This is really great, I really appreciate the quick fix! As you said, it now pulls TF2.3 and performs the conversion.

However, while testing it, I noticed that the tensor shape is different from my workaround method.

SavedModel via workaround:

$ saved_model_cli show --dir /posenet/saved_model_workaround --all

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['sub_2'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, -1, -1, 3)
        name: sub_2:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['float_heatmaps'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, -1, -1, 17)
        name: float_heatmaps:0
    outputs['float_short_offsets'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, -1, -1, 34)
        name: float_short_offsets:0
  Method name is: tensorflow/serving/predict

SavedModel via tfjs-graph-converter:

$ saved_model_cli show --dir /posenet/saved_model_converter --all

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['sub_2:0'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, -1, -1, 3)
        name: sub_2:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['float_heatmaps:0'] tensor_info:
        dtype: DT_FLOAT
        shape: (17)
        name: Const_114:0
    outputs['float_short_offsets:0'] tensor_info:
        dtype: DT_FLOAT
        shape: (34)
        name: Const_112:0
    outputs['resnet_v1_50/displacement_bwd_2/BiasAdd:0'] tensor_info:
        dtype: DT_FLOAT
        shape: (32)
        name: Const_110:0
    outputs['resnet_v1_50/displacement_fwd_2/BiasAdd:0'] tensor_info:
        dtype: DT_FLOAT
        shape: (32)
        name: Const:0
  Method name is: tensorflow/serving/predict

For example, when I perform inference on a sample image which has 1280x720 pixels using PoseNet with stride 16, the response from model produced by tfjs-graph-converter is quite different (significant part of information is missing so it is not possible to determine positions of the keypoints).

Model tf_result_proto.ByteSize() float_heatmaps.shape float_short_offsets.shape
saved_model_workaround 734533 (1, 45, 80, 17) (1, 45, 80, 34)
saved_model_converter 704 (17) (34)

I don't know if this only affects PoseNet models or is a more broad issue. Just wanted to bring it to your attention.

bhavikapanara commented 3 years ago

Additional comment: if you want to continue to convert the result to TFLite (which I did), then the following code snippet from the converter API guide may be helpful (this can basically be appended right below the code example from @glenvorel ):

model = tf.saved_model.load(export_dir)
concrete_func = model.signatures[
  tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY]
concrete_func.inputs[0].set_shape([1, 256, 256, 3])
converter = TFLiteConverter.from_concrete_functions([concrete_func])

Thanks, @floe, Using this code I can able to generate TFLite model. But how to decode heatmap and offset in order to generate detected keypoints.

Here is the output details of the TFLite model:

interpreter.get_output_details()
output_details
[{'name': 'float_heatmaps',
  'index': 195,
  'shape': array([ 1, 16, 16, 17], dtype=int32),
  'shape_signature': array([ 1, 16, 16, 17], dtype=int32),
  'dtype': numpy.float32,
  'quantization': (0.0, 0),
  'quantization_parameters': {'scales': array([], dtype=float32),
   'zero_points': array([], dtype=int32),
   'quantized_dimension': 0},
  'sparsity_parameters': {}},
 {'name': 'float_short_offsets',
  'index': 196,
  'shape': array([ 1, 16, 16, 34], dtype=int32),
  'shape_signature': array([ 1, 16, 16, 34], dtype=int32),
  'dtype': numpy.float32,
  'quantization': (0.0, 0),
  'quantization_parameters': {'scales': array([], dtype=float32),
   'zero_points': array([], dtype=int32),
   'quantized_dimension': 0},
  'sparsity_parameters': {}},
 {'name': 'resnet_v1_50/displacement_bwd_2/BiasAdd',
  'index': 197,
  'shape': array([ 1, 16, 16, 32], dtype=int32),
  'shape_signature': array([ 1, 16, 16, 32], dtype=int32),
  'dtype': numpy.float32,
  'quantization': (0.0, 0),
  'quantization_parameters': {'scales': array([], dtype=float32),
   'zero_points': array([], dtype=int32),
   'quantized_dimension': 0},
  'sparsity_parameters': {}},
 {'name': 'resnet_v1_50/displacement_fwd_2/BiasAdd',
  'index': 198,
  'shape': array([ 1, 16, 16, 32], dtype=int32),
  'shape_signature': array([ 1, 16, 16, 32], dtype=int32),
  'dtype': numpy.float32,
  'quantization': (0.0, 0),
  'quantization_parameters': {'scales': array([], dtype=float32),
   'zero_points': array([], dtype=int32),
   'quantized_dimension': 0},
  'sparsity_parameters': {}}]

Please help me to get the dimension of the detected key-points along with a score.

Thanks, Bhavika

floe commented 3 years ago

I've never used the keypoints, just the heatmap - sorry. I guess you'd have to look in the body-pix JS demo code.

bhavikapanara commented 3 years ago

I've never used the keypoints, just the heatmap - sorry. I guess you'd have to look in the body-pix JS demo code.

Thanks, @floe for your response. But How you have used heatmap?

floe commented 3 years ago

I only used the output node float_segments and classified everything above 0.65 as body part, nothing more.