search

onnx / keras-onnx

Convert tf.keras/Keras models to ONNX
Apache License 2.0
379 stars 109 forks source link

The generated ONNX model needs run with the custom op supports. #586

Open bedapisl opened 4 years ago

bedapisl commented 4 years ago

Hello, I am trying to convert my Keras model to ONNX format, but I am getting lots of warnings and an error:

WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/TensorArrayStack/TensorArrayGatherV3 of type TensorArrayGatherV3
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/TensorArray_1 of type TensorArrayV3
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/Exit_2 of type Exit
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/TensorArrayStack/TensorArraySizeV3 of type TensorArraySizeV3
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/Switch_2 of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/Merge_2 of type Merge
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/LoopCond of type LoopCond
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/Enter_2 of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/NextIteration_2 of type NextIteration
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/cond/Merge of type Merge
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/cond/Merge_1 of type Merge
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/cond/Merge_2 of type Merge
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/cond/Merge_3 of type Merge
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/TensorArrayWrite/TensorArrayWriteV3 of type TensorArrayWriteV3
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/TensorArrayWrite/TensorArrayWriteV3/Enter of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/cond/Merge of type Merge
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/Merge of type Merge
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/Less/Enter of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/Merge_1 of type Merge
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/cond/ExpandDims_4/Switch of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/cond/ExpandDims_5/Switch of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/cond/ExpandDims_6/Switch of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/cond/ExpandDims_7/Switch of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/Switch_1 of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/Enter of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/NextIteration of type NextIteration
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/Enter_1 of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/NextIteration_1 of type NextIteration
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/cond/mul/Switch of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/cond/mul/Switch_1 of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/cond/mul_1/Switch of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/cond/mul_1/Switch_1 of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/cond/mul_2/Switch of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/cond/mul_3/Switch of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/cond/strided_slice/Switch of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/cond/Shape/Switch of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/cond/sub/Switch of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/cond/TopKV2/Switch of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/GreaterEqual/Enter of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/TensorArrayStack/TensorArrayGatherV3 of type TensorArrayGatherV3
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/Switch of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/TensorArray_1 of type TensorArrayV3
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/Exit_2 of type Exit
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/TensorArrayStack/TensorArraySizeV3 of type TensorArraySizeV3
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/Switch_2 of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/Merge_2 of type Merge
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/LoopCond of type LoopCond
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/Enter_2 of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/NextIteration_2 of type NextIteration
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/TensorArrayWrite/TensorArrayWriteV3 of type TensorArrayWriteV3
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/TensorArrayWrite/TensorArrayWriteV3/Enter of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/Merge of type Merge
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/Less/Enter of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/Merge_1 of type Merge
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/Switch_1 of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/cond/Merge of type Merge
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/Enter of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/NextIteration of type NextIteration
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/Enter_1 of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/NextIteration_1 of type NextIteration
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/cond/strided_slice/Switch of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/cond/non_maximum_suppresion/NonMaxSuppressionV3/Switch of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/cond/non_maximum_suppresion/NonMaxSuppressionV3/Enter_1 of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/Switch of type Switch
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/cond/non_maximum_suppresion/NonMaxSuppressionV3/Enter of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/strided_slice/Enter of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/TensorArrayReadV3 of type TensorArrayReadV3
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/TensorArrayReadV3 of type TensorArrayReadV3
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/Greater/Enter_1 of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/TensorArrayReadV3/Enter of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/TensorArrayReadV3/Enter_1 of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/TensorArrayReadV3/Enter of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/TensorArrayReadV3/Enter_1 of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/while/Greater/Enter of type Enter
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/TensorArray of type TensorArrayV3
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/while/loop_over_classes/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3 of type TensorArrayScatterV3
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/TensorArray of type TensorArrayV3
      The generated ONNX model needs run with the custom op supports.
WARN: No corresponding ONNX op matches the tf.op node decoded_predictions/loop_over_batch/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3 of type TensorArrayScatterV3
      The generated ONNX model needs run with the custom op supports.
Traceback (most recent call last):
  File "onnx_test.py", line 11, in <module>
    onnx_model = onnxmltools.convert_keras(keras_model)
  File "/home/pislb/packages/onnxmltools/onnxmltools/convert/main.py", line 33, in convert_keras
    return convert(model, name, doc_string, target_opset, channel_first_inputs)
  File "/home/pislb/.local/lib/python3.6/site-packages/keras2onnx/main.py", line 80, in convert_keras
    parse_graph(topology, tf_graph, target_opset, output_names, output_dict)
  File "/home/pislb/.local/lib/python3.6/site-packages/keras2onnx/parser.py", line 842, in parse_graph
    graph, keras_node_dict, topo, top_level, output_names)
  File "/home/pislb/.local/lib/python3.6/site-packages/keras2onnx/parser.py", line 606, in _parse_graph_core
    _on_parsing_tf_nodes(graph, nodes, varset, topology.debug_mode)
  File "/home/pislb/.local/lib/python3.6/site-packages/keras2onnx/parser.py", line 323, in _on_parsing_tf_nodes
    var_type = infer_variable_type(i_, varset.target_opset)
  File "/home/pislb/.local/lib/python3.6/site-packages/keras2onnx/_parser_tf.py", line 48, in infer_variable_type
    "Unable to find out a correct type for tensor type = {} of {}".format(tensor_type, tensor.name))
ValueError: Unable to find out a correct type for tensor type = 0 of decoded_predictions/loop_over_batch/while/TensorArrayWrite/TensorArrayWriteV3/Enter:0

Does this mean I have to reimplement all the ops mentioned in the warnings in ONNX? And how do I fix the error? Thanks for help

(I also posted this to onnxmltools (https://github.com/onnx/onnxmltools/issues/411).

wenbingl commented 4 years ago

If there are too many tf.ops in the error message, probably there are some layers were not able to be converted. can you share more details about this model?

bedapisl commented 4 years ago

Hello, the model is SSD network from https://github.com/pierluigiferrari, the problematic layer is this one: https://github.com/pierluigiferrari/ssd_keras/blob/master/keras_layers/keras_layer_DecodeDetections.py#L27

bedapisl commented 4 years ago

Hello, here is a simple example which demonstrates the problem:

import keras

import sys
sys.path.insert(0, "./ssd_keras")  #path to https://github.com/pierluigiferrari/ssd_keras

from keras_layers.keras_layer_DecodeDetections import DecodeDetections

import onnxmltools

output_onnx_model = 'model.onnx'

keras_model = keras.Sequential()
keras_model.add(DecodeDetections(img_height=512, img_width=512))
keras_model.build(input_shape=(None, 24564, 98))

onnx_model = onnxmltools.convert_keras(keras_model, target_opset=12)

onnxmltools.utils.save_model(onnx_model, output_onnx_model)

To run it you will have to clone https://github.com/pierluigiferrari/ssd_keras