search

PINTO0309 / openvino2tensorflow

This script converts the ONNX/OpenVINO IR model to Tensorflow's saved_model, tflite, h5, tfjs, tftrt(TensorRT), CoreML, EdgeTPU, ONNX and pb. PyTorch (NCHW) -> ONNX (NCHW) -> OpenVINO (NCHW) -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TFLite (NHWC/NCHW). And the conversion from .pb to saved_model and from saved_model to .pb and from .pb to .tflite and saved_model to .tflite and saved_model to onnx. Support for building environments with Docker. It is possible to directly access the host PC GUI and the camera to verify the operation. NVIDIA GPU (dGPU) support. Intel iHD GPU (iGPU) support.
MIT License
338 stars 39 forks source link

ValueError: Dimensions must be equal. NCHW v NHWC inconsistency #29

Closed zshn25 closed 3 years ago

zshn25 commented 3 years ago

1. OS you are using e.g. Ubuntu 20.04, WIndows10, etc

Ubuntu 18.04

2. OS Architecture e.g. x86_64, armv7l, aarch64, etc

x86 64

3. Version of OpenVINO e.g. 2021.2.185, etc

2021.2.0-1877-176bdf51370

4. Version of TensorFlow e.g. v2.4.1, tf-nightly==2.5.0.dev20210128, etc

2.4.1

8. Version of ONNX e.g. v1.8.0, etc

1.7.0

9. Download URL for ONNX model

my onnx model is located here

For OpenVino conversion, I did

python /opt/intel/openvino_2021/deployment_tools/model_optimizer/mo.py --input_model resnet18.onnx --input_shape [1,3,192,640] --output_dir openvino/192x640/FP32 --data_type FP32

and then for TFLite, I did 

openvino2tensorflow \
  --model_path openvino/192x640/FP32/resnet18.xml \
  --model_output_path resnet18 \
  --output_saved_model True \
  --output_edgetpu=True

13. Issue Details

I get the following error. I think there is some inconsistency while NCHWto NHWCconversion.

TensorFlow/Keras model building process starts ======================================
Traceback (most recent call last):
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 332, in add
    x, y, name=name, ctx=_ctx)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 366, in add_eager_fallback
    _attr_T, _inputs_T = _execute.args_to_matching_eager([x, y], ctx, [_dtypes.bfloat16, _dtypes.half, _dtypes.float32, _dtypes.float64, _dtypes.uint8, _dtypes.int8, _dtypes.int16, _dtypes.int32, _dtypes.int64, _dtypes.complex64, _dtypes.complex128, _dtypes.string, ])
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/eager/execute.py", line 265, in args_to_matching_eager
    tensor = ops.convert_to_tensor(t, ctx=ctx)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/profiler/trace.py", line 163, in wrapped
    return func(*args, **kwargs)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1540, in convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py", line 339, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py", line 265, in constant
    allow_broadcast=True)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py", line 276, in _constant_impl
    return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py", line 301, in _constant_eager_impl
    t = convert_to_eager_tensor(value, ctx, dtype)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py", line 98, in convert_to_eager_tensor
    return ops.EagerTensor(value, ctx.device_name, dtype)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/keras/engine/keras_tensor.py", line 274, in __array__
    'Cannot convert a symbolic Keras input/output to a numpy array. '
TypeError: Cannot convert a symbolic Keras input/output to a numpy array. This error may indicate that you're trying to pass a symbolic value to a NumPy call, which is not supported. Or, you may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1853, in _create_c_op
    c_op = pywrap_tf_session.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Dimensions must be equal, but are 1920 and 3 for '{{node tf.math.add_55/Add}} = Add[T=DT_FLOAT](Placeholder, tf.math.add_55/Add/y)' with input shapes: [1,1920,85,3], [1,3,1920,85].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/e2r/Desktop/e2r/easy2ride_pipeline/src/openvino2tensorflow/scripts/openvino2tensorflow", line 478, in convert
    tf_layers_dict[layer_id] = tf.math.add(tf_layers_dict[edge_id0], tf_layers_dict[edge_id1])
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 337, in add
    add, (), dict(x=x, y=y, name=name)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py", line 122, in dispatch
    result = dispatcher.handle(op, args, kwargs)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py", line 1450, in handle
    return TFOpLambda(op)(*args, **kwargs)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 952, in __call__
    input_list)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1091, in _functional_construction_call
    inputs, input_masks, args, kwargs)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 822, in _keras_tensor_symbolic_call
    return self._infer_output_signature(inputs, args, kwargs, input_masks)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 863, in _infer_output_signature
    outputs = call_fn(inputs, *args, **kwargs)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py", line 1327, in _call_wrapper
    return self._call_wrapper(*args, **kwargs)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py", line 1359, in _call_wrapper
    result = self.function(*args, **kwargs)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 345, in add
    "Add", x=x, y=y, name=name)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py", line 750, in _apply_op_helper
    attrs=attr_protos, op_def=op_def)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 592, in _create_op_internal
    compute_device)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3536, in _create_op_internal
    op_def=op_def)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 2016, in __init__
    control_input_ops, op_def)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1856, in _create_c_op
    raise ValueError(str(e))
ValueError: Dimensions must be equal, but are 1920 and 3 for '{{node tf.math.add_55/Add}} = Add[T=DT_FLOAT](Placeholder, tf.math.add_55/Add/y)' with input shapes: [1,1920,85,3], [1,3,1920,85].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 332, in add
    x, y, name=name, ctx=_ctx)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 366, in add_eager_fallback
    _attr_T, _inputs_T = _execute.args_to_matching_eager([x, y], ctx, [_dtypes.bfloat16, _dtypes.half, _dtypes.float32, _dtypes.float64, _dtypes.uint8, _dtypes.int8, _dtypes.int16, _dtypes.int32, _dtypes.int64, _dtypes.complex64, _dtypes.complex128, _dtypes.string, ])
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/eager/execute.py", line 265, in args_to_matching_eager
    tensor = ops.convert_to_tensor(t, ctx=ctx)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/profiler/trace.py", line 163, in wrapped
    return func(*args, **kwargs)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1540, in convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py", line 339, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py", line 265, in constant
    allow_broadcast=True)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py", line 276, in _constant_impl
    return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py", line 301, in _constant_eager_impl
    t = convert_to_eager_tensor(value, ctx, dtype)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py", line 98, in convert_to_eager_tensor
    return ops.EagerTensor(value, ctx.device_name, dtype)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/keras/engine/keras_tensor.py", line 274, in __array__
    'Cannot convert a symbolic Keras input/output to a numpy array. '
TypeError: Cannot convert a symbolic Keras input/output to a numpy array. This error may indicate that you're trying to pass a symbolic value to a NumPy call, which is not supported. Or, you may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1853, in _create_c_op
    c_op = pywrap_tf_session.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Dimensions must be equal, but are 1920 and 3 for '{{node tf.math.add_56/Add}} = Add[T=DT_FLOAT](Placeholder, tf.math.add_56/Add/y)' with input shapes: [1,1920,85,3], [1,3,1920,85].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/e2r/anaconda3/envs/e2r/bin/openvino2tensorflow", line 7, in <module>
    exec(compile(f.read(), __file__, 'exec'))
  File "/home/e2r/Desktop/e2r/easy2ride_pipeline/src/openvino2tensorflow/scripts/openvino2tensorflow", line 2366, in <module>
    main()
  File "/home/e2r/Desktop/e2r/easy2ride_pipeline/src/openvino2tensorflow/scripts/openvino2tensorflow", line 2362, in main
    yolact, weight_replacement_config, debug, debug_layer_number)
  File "/home/e2r/Desktop/e2r/easy2ride_pipeline/src/openvino2tensorflow/scripts/openvino2tensorflow", line 483, in convert
    tf_layers_dict[layer_id] = tf.math.add(tf_layers_dict[edge_id0], tf_layers_dict[edge_id1])
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 337, in add
    add, (), dict(x=x, y=y, name=name)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py", line 122, in dispatch
    result = dispatcher.handle(op, args, kwargs)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py", line 1450, in handle
    return TFOpLambda(op)(*args, **kwargs)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 952, in __call__
    input_list)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1091, in _functional_construction_call
    inputs, input_masks, args, kwargs)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 822, in _keras_tensor_symbolic_call
    return self._infer_output_signature(inputs, args, kwargs, input_masks)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 863, in _infer_output_signature
    outputs = call_fn(inputs, *args, **kwargs)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py", line 1327, in _call_wrapper
    return self._call_wrapper(*args, **kwargs)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py", line 1359, in _call_wrapper
    result = self.function(*args, **kwargs)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 345, in add
    "Add", x=x, y=y, name=name)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py", line 750, in _apply_op_helper
    attrs=attr_protos, op_def=op_def)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 592, in _create_op_internal
    compute_device)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3536, in _create_op_internal
    op_def=op_def)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 2016, in __init__
    control_input_ops, op_def)
  File "/home/e2r/anaconda3/envs/e2r/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1856, in _create_c_op
    raise ValueError(str(e))
ValueError: Dimensions must be equal, but are 1920 and 3 for '{{node tf.math.add_56/Add}} = Add[T=DT_FLOAT](Placeholder, tf.math.add_56/Add/y)' with input shapes: [1,1920,85,3], [1,3,1920,85].
PINTO0309 commented 3 years ago

The transposition of weights in Add OP was insufficient, so the process was added and the error was corrected so that it does not occur. 264ebfe6c6519c98da83333f6e5f19d6a8b3d40d openvino2tensorflow==v1.10.2

The conversion will be successful, but there are some things you need to be aware of. Currently, it is not possible to correctly predict the shape of a 5D Transpose or Reshape. Therefore, you need to follow the steps here and use an external file (json) to force the correction of incorrect transposition behavior. 6-7. Replace weights or constant values in Const OP

Note that EdgeTPU does not support 5D transpose operation.

PINTO0309 commented 3 years ago

Closed due to lack of progress.