Success：Pytoch - > onnx - > tensorflow ，However, when using tensorflow for inference, the output is empty !!!

[tensorflowt]

Describe the bug

Hello! I recently trained the Efficientdet model through the pytorch framework. During the training process, I saved the training model as.onnx model, and then I implemented the.onnx model to.pb model through the script.

But when I use the tensorflow framework for inference, the output is empty.

My code is as follows:

To Reproduce

Readme:

• python onnx2tf.py

  
  # onnx2tf.py
  import cv2
  import numpy as np
  import onnx
  from onnx_tf.backend import prepare

model = onnx.load('./signatrix_efficientdet_coco_0220.onnx') img = cv2.imread("./car.jpg") img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB).astype(np.float32) img = cv2.resize(img,(512,512)) img = img[np.newaxis, ...,] img = np.transpose(img, [0,3,1,2])

tf_rep = prepare(model) output1,output2,output3 = tf_rep.run(img.reshape([1,3,512,512])) print(output1) tf_rep.export_graph('tf.pb')

- python inference.py

inference.py

import cv2 import numpy as np import tensorflow as tf from tensorflow.python.platform import gfile

model = "tf.pb" img = cv2.imread("./car.jpg") img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB).astype(np.float32) img = cv2.resize(img,(512,512)) img = img[np.newaxis, ...,] img = np.transpose(img, [0,3,1,2])

with tf.Session() as sess: print("load graph") with gfile.FastGFile(model, 'rb') as f: graph_def = tf.GraphDef() graph_def.ParseFromString(f.read())

sess.graph.as_default()
tf.import_graph_def(graph_def, name='')

inp  = sess.graph.get_tensor_by_name('inputs:0')
out1 = sess.graph.get_tensor_by_name('539:0')
out2 = sess.graph.get_tensor_by_name('540:0')
out3 = sess.graph.get_tensor_by_name('541:0')
out  = [out1,out2,out3]
feed_dict = {inp: img}

result1,result2,result3 = sess.run(out,feed_dict)
print(result1)
print(result1.shape)
print(result2.shape)
print(result3.shape)

result:

[]

(0,)

(0,)

(0, 4)

**ONNX model file**

[signatrix_efficientdet_coco_0220.zip](https://github.com/onnx/onnx-tensorflow/files/4229461/signatrix_efficientdet_coco_0220.zip)

**Python, ONNX, ONNX-TF, Tensorflow version**

This section can be obtained by running `get_version.py` from util folder.
 - Python version: 3.6.10
 - ONNX version: 1.6.0
 - ONNX-TF version:1.5.0
 - Tensorflow version: 1.14.0

**Additional context**

![car](https://user-images.githubusercontent.com/37217594/74912991-b44b8900-53fa-11ea-84ed-5d6c76146c90.jpg)

[sdmonov]

@tensorflowt when I looked at the onnx model I can see only three Constant operators which produce empty outputs, with shapes (0,), (0,) and (1, 4). Please take a look at the attachment. Looking into the exported tensorflow model I can see the same. Can you please check the code in pytorch that exports the model. Could be something missing there.

[tensorflowt]

@tensorflowt when I looked at the onnx model I can see only three Constant operators which produce empty outputs, with shapes (0,), (0,) and (1, 4). Please take a look at the attachment. Looking into the exported tensorflow model I can see the same. Can you please check the code in pytorch that exports the model. Could be something missing there.

Thank you very much for taking time out of your busy schedule to reply.

Yesterday, I was also wondering whether it was because of the onnx model generated by pytorch, but I tested other nodes of the pb model file today and found that the data could be output normally. The details are as follows:

The following are the specific documents:

model.txt result1.txt result2.txt

So, what I want to make sure is, am I just having a problem with the choice of the output node, or am I really having a problem with the model transformation?

Thank you very much ！ Best wishes！

[sdmonov]

@tensorflowt looks like the onnx file contains only the learned model parameters saved as initializers, which are converted to tf.constant in Tensorflow. There are only 3 nodes in the model and they are Constant too. Looks like the export from PyTorch did not export the model itself but only the learned parameters. Maybe you can try verbose=True when calling torch.onnx.export to see if the TorchScript compiler traced the model correctly. I'm not familiar with this model implementation but you may need to use script-based exporter (check https://pytorch.org/docs/stable/onnx.html#tracing-vs-scripting and https://pytorch.org/docs/stable/jit.html).

Hope this helps. Best regards!

[tensorflowt]

Thank you very much for your suggestion. I set the training parameter verbose to True here. The results printed from the model are as follows: onnx.txt I have compared the model information of the different frameworks (onnx and tf), which is basically the same. Therefore, I wonder if it is because the output node of my model is not selected correctly?

Thank you very much ！ Best wishes！

[MaratZakirov]

Hi, have you ever need to solve HCWH (pytorch native format) to HWHC (TF preferable and TFlite native) mistmatch issue?

[sdmonov]

@MaratZakirov I guess you meant NCHW (PyTorch and cuDNN default) vs NHWC (used by TF). NCHW is the input format for ONNX too. TF supports NCHW format with the data_format argument but this argument may not work on CPU depending how your TF was compiled and what architecture it is running (e.g. using mkl on Intel or not). Some of the onnx-tensorflow converter operators will try to use NCHW if possible or they will do a tf.transpose of the data to convert it to NHWC.

[MaratZakirov]

@MaratZakirov I guess you meant NCHW (PyTorch and cuDNN default) vs NHWC (used by TF). NCHW is the input format for ONNX too. TF supports NCHW format with the data_format argument but this argument may not work on CPU depending how your TF was compiled and what architecture it is running (e.g. using mkl on Intel or not). Some of the onnx-tensorflow converter operators will try to use NCHW if possible or they will do a tf.transpose of the data to convert it to NHWC.

I actually need TF just to convert to TFlite. In my case I do not see that onnx-tensorflow did convert some of my NCWH convs to NWHC. For that purpose I try to use special converter which transposes convs to NWHC format https://github.com/paulbauriegel/tensorflow-tools/blob/master/convert-model-to-NHWC.py and with it I also have somne problems due to TFlite do not have Dilation2D ops but some how onnx-tensorflow put em into pb file

[mgarbade]

As far as my experience goes the 'NHWC problem' goes away when updating to tf 2.x as mentioned in #617 . However some other conversion issues remain in my case