TensorRT benchmark error

[xiaopojizu123]

I clone the code on TX2, and i use the following command: Tensorflow 2.1 CUDA 10.0 TensorRT 6.0.1.10

1. $ python3 model_inspect.py --runmode=saved_model --model_name=efficientdet-d0 -ckpt_path=efficientdet-d0 --saved_model_dir=/tmp/saved_model --tensorrt=FP32
2. $ python3 model_inspect.py --runmode=saved_model_benchmark --saved_model_dir=/tmp/saved_model/tensorrt_fp32 --model_name=efficientdet-d0 --input_image=datasets/image1.bmp --output_image_dir=/tmp/result

and then i got the follow error:

2020-06-24 09:52:54.015065: W tensorflow/core/framework/op_kernel.cc:1655] OP_REQUIRES failed at trt_engine_op.cc:492 : Invalid argument: Input shapes are inconsistent on the batch dimension, for TRTEngineOp_28: [[0,4], [1], [1], [1], [1], [0], [0], [0]]
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1367, in _do_call
    return fn(*args)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1352, in _run_fn
    target_list, run_metadata)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1445, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument: Input shapes are inconsistent on the batch dimension, for TRTEngineOp_28: [[0,4], [1], [1], [1], [1], [0], [0], [0]]
     [[{{node TRTEngineOp_28}}]]
  (1) Invalid argument: Input shapes are inconsistent on the batch dimension, for TRTEngineOp_28: [[0,4], [1], [1], [1], [1], [0], [0], [0]]
     [[{{node TRTEngineOp_28}}]]
     [[detections/_51]]
0 successful operations.
0 derived errors ignored.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "model_inspect.py", line 490, in <module>
    app.run(main)
  File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 299, in run
    _run_main(main, args)
  File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 250, in _run_main
    sys.exit(main(argv))
  File "model_inspect.py", line 484, in main
    trace_filename=FLAGS.trace_filename)
  File "model_inspect.py", line 425, in run_model
    trace_filename=kwargs.get('trace_filename', None))
  File "model_inspect.py", line 205, in saved_model_benchmark
    driver.benchmark(raw_images, trace_filename)
  File "/home/nvidia/Downloads/automl-master/efficientdet/inference.py", line 625, in benchmark
    feed_dict={self.signitures['image_arrays']: image_arrays})
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 960, in run
    run_metadata_ptr)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1183, in _run
    feed_dict_tensor, options, run_metadata)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1361, in _do_run
    run_metadata)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1386, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument: Input shapes are inconsistent on the batch dimension, for TRTEngineOp_28: [[0,4], [1], [1], [1], [1], [0], [0], [0]]
     [[node TRTEngineOp_28 (defined at /home/nvidia/Downloads/automl-master/efficientdet/inference.py:681) ]]
  (1) Invalid argument: Input shapes are inconsistent on the batch dimension, for TRTEngineOp_28: [[0,4], [1], [1], [1], [1], [0], [0], [0]]
     [[node TRTEngineOp_28 (defined at /home/nvidia/Downloads/automl-master/efficientdet/inference.py:681) ]]
     [[detections/_51]]
0 successful operations.
0 derived errors ignored.

also ,i use the saved_model_dir=/tmp/saved_model,it can work. How can i solve this problem? Thanks!!!

[xiaopojizu123]

Can someone help me？

[xiaopojizu123]

@mingxingtan can you help me ? thanks a lot！

[mingxingtan]

@xiaopojizu123 Yeah, looks like there is some mismatch between TensorRT and TF. It runs in colab, but fails for some customized TF+TRT version. See #460 Sorry, I also don't have a good solution yet.

[xiaopojizu123]

@xiaopojizu123 Yeah, looks like there is some mismatch between TensorRT and TF. It runs in colab, but fails for some customized TF+TRT version. See #460 Sorry, I also don't have a good solution yet.

Okay, I will try some other versions.Thanks for your reply