Snixells commented 4 years ago

Prerequisites

Please answer the following questions for yourself before submitting an issue.

[X] I am using the latest TensorFlow Model Garden release and TensorFlow 2.
[X] I am reporting the issue to the correct repository. (Model Garden official or research directory)
[X] I checked to make sure that this issue has not already been filed.

1. The entire URL of the file you are using

https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md

2. Describe the bug

Hey there! I am trying to convert the SSD ResNet50 V1 FPN 640x640 (RetinaNet50) of the new Tensorflow 2 Object Detection API to a TensorRT Model to run on my Jetson AGX Board.

3. Steps to reproduce

I am running the following Code:

import tensorflow as tf
import numpy as np
from tensorflow.python.compiler.tensorrt import trt_convert as trt

input_saved_model_dir = './ssd_resnet50_v1_fpn_640x640_coco17_tpu-8/saved_model/'
output_saved_model_dir = './models/tensorRT/'
num_runs = 2

conversion_params = trt.DEFAULT_TRT_CONVERSION_PARAMS
conversion_params = conversion_params._replace(max_workspace_size_bytes=(1<<32))
conversion_params = conversion_params._replace(precision_mode="FP16")
# conversion_params = conversion_params._replace(maximum_cached_engiens=100)

converter = trt.TrtGraphConverterV2(input_saved_model_dir=input_saved_model_dir,conversion_params=conversion_params)
converter.convert()

def my_input_fn():
    for _ in range(num_runs):
        inp1 = np.random.normal(size=(1, 640, 640, 3)).astype(np.uint8)
        yield inp1

converter.build(input_fn=my_input_fn)
converter.save(output_saved_model_dir)

When running that Code I am getting the following Error:

InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-7-d7c3941a6051> in <module>
      7         yield inp1
      8 
----> 9 converter.build(input_fn=my_input_fn)
     10 converter.save(output_saved_model_dir)

/projects/sebschaefer/venv/tf22gpu_copy/lib/python3.6/site-packages/tensorflow/python/compiler/tensorrt/trt_convert.py in build(self, input_fn)
   1172       if not first_input:
   1173         first_input = inp
-> 1174       func(*map(ops.convert_to_tensor, inp))
   1175 
   1176     if self._need_trt_profiles:

/projects/sebschaefer/venv/tf22gpu_copy/lib/python3.6/site-packages/tensorflow/python/eager/function.py in __call__(self, *args, **kwargs)
   1603       TypeError: For invalid positional/keyword argument combinations.
   1604     """
-> 1605     return self._call_impl(args, kwargs)
   1606 
   1607   def _call_impl(self, args, kwargs, cancellation_manager=None):

/projects/sebschaefer/venv/tf22gpu_copy/lib/python3.6/site-packages/tensorflow/python/eager/function.py in _call_impl(self, args, kwargs, cancellation_manager)
   1643       raise TypeError("Keyword arguments {} unknown. Expected {}.".format(
   1644           list(kwargs.keys()), list(self._arg_keywords)))
-> 1645     return self._call_flat(args, self.captured_inputs, cancellation_manager)
   1646 
   1647   def _filtered_call(self, args, kwargs):

/projects/sebschaefer/venv/tf22gpu_copy/lib/python3.6/site-packages/tensorflow/python/eager/function.py in _call_flat(self, args, captured_inputs, cancellation_manager)
   1744       # No tape is watching; skip to running the function.
   1745       return self._build_call_outputs(self._inference_function.call(
-> 1746           ctx, args, cancellation_manager=cancellation_manager))
   1747     forward_backward = self._select_forward_and_backward_functions(
   1748         args,

/projects/sebschaefer/venv/tf22gpu_copy/lib/python3.6/site-packages/tensorflow/python/eager/function.py in call(self, ctx, args, cancellation_manager)
    596               inputs=args,
    597               attrs=attrs,
--> 598               ctx=ctx)
    599         else:
    600           outputs = execute.execute_with_cancellation(

/projects/sebschaefer/venv/tf22gpu_copy/lib/python3.6/site-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     58     ctx.ensure_initialized()
     59     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 60                                         inputs, attrs, num_outputs)
     61   except core._NotOkStatusException as e:
     62     if name is not None:

InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument:  Input shape axis 0 must equal 1, got shape [640,640,3]
     [[node StatefulPartitionedCall/Preprocessor/unstack (defined at <ipython-input-7-d7c3941a6051>:2) ]]
     [[StatefulPartitionedCall/Postprocessor/BatchMultiClassNonMaxSuppression/TRTEngineOp_11/_106]]
  (1) Invalid argument:  Input shape axis 0 must equal 1, got shape [640,640,3]
     [[node StatefulPartitionedCall/Preprocessor/unstack (defined at <ipython-input-7-d7c3941a6051>:2) ]]
0 successful operations.
0 derived errors ignored. [Op:__inference_pruned_176027]

Function call stack:
pruned -> pruned

The error says that there is a Problem with my Input Dimensions. It specificially says that my first (index 0) dimension needs to be 1. I am passing a numpy array with the first Dimension beeing 1 (1, 640, 640, 3), but in the Error message it says that the Array is of shape [640,640,3].

I am not sure how to change my input so that it satisfies the requirements.

Thanks for your help!

4. Expected behavior

I want a TensorRT optimized Model so that I can run it on my Jetson AGX

6. System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
TensorFlow version (use command below): 2.2
Python version: 3.7

ravikyram commented 4 years ago

I have tried in colab with TF nightly version(2.4.0-dev20200803) and i am seeing session is crashed.Please, find the gist here.Thanks!

abhi-kumar commented 4 years ago

That input size error Input shape axis 0 must equal 1, got shape [640,640,3] was resolved with

inp1 = np.random.normal(size=(1, 1, 640, 640, 3)).astype(np.uint8)

Or you can use

inp1 = tf.cast(tf.random.uniform((1, 1, 640, 640, 3)), tf.dtypes.uint8)

Still TRT conversion didn't speed up things much. I am yet to understand which operators are not supported yet.

tensorflow / models