Open mankeyboy opened 4 years ago
@pooyadavoodi Any guidance here would be helpful :)
Looks like the output of the graph (i.e. graph_func(batch_images)
) is a list instead of a tensor. Check how many items are in the list. My guess is that the output tensor is the first item in the list. If that's the case, you can do something like this: batch_preds = graph_func(batch_images)[0].values().numpy()
Thanks @pooyadavoodi . Your suggestion worked and I was able to succesfully run the synthetic test for batch size 128. However, when I tried to run it next for batch size 1, the same line in the code gave this error:
2020-01-08 22:34:20.448568: W tensorflow/core/common_runtime/base_collective_executor.cc:217] BaseCollectiveExecutor::StartAbort Invalid argument: Number of ways to split should evenly divide the split dimension, but got split_dim 0 (size = 1) and num_split 2
[[{{node PartitionedCall/split_inputs/split}}]]
[[PartitionedCall/split_inputs/split/_2]]
2020-01-08 22:34:20.448681: W tensorflow/core/common_runtime/base_collective_executor.cc:217] BaseCollectiveExecutor::StartAbort Invalid argument: Number of ways to split should evenly divide the split dimension, but got split_dim 0 (size = 1) and num_split 2
[[{{node PartitionedCall/split_inputs/split}}]]
2020-01-08 22:34:20.449397: W tensorflow/core/common_runtime/base_collective_executor.cc:217] BaseCollectiveExecutor::StartAbort Invalid argument: Number of ways to split should evenly divide the split dimension, but got split_dim 0 (size = 1) and num_split 2
[[{{node PartitionedCall/split_inputs/split}}]]
[[PartitionedCall/TRTEngineOp_0/_6]]
Traceback (most recent call last):
File "image_classification.py", line 462, in <module>
target_duration=args.target_duration)
File "image_classification.py", line 256, in run_inference
batch_preds = graph_func(batch_images)[0].numpy()
File "/home/mayroy13/anaconda3/envs/trt7-py36/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py", line 1551, in __call__
return self._call_impl(args, kwargs)
File "/home/mayroy13/anaconda3/envs/trt7-py36/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py", line 1591, in _call_impl
return self._call_flat(args, self.captured_inputs, cancellation_manager)
File "/home/mayroy13/anaconda3/envs/trt7-py36/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py", line 1692, in _call_flat
ctx, args, cancellation_manager=cancellation_manager))
File "/home/mayroy13/anaconda3/envs/trt7-py36/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py", line 545, in call
ctx=ctx)
File "/home/mayroy13/anaconda3/envs/trt7-py36/lib/python3.6/site-packages/tensorflow_core/python/eager/execute.py", line 67, in quick_execute
six.raise_from(core._status_to_exception(e.code, message), None)
File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: Number of ways to split should evenly divide the split dimension, but got split_dim 0 (size = 1) and num_split 2
[[node PartitionedCall/split_inputs/split (defined at image_classification.py:134) ]]
[[PartitionedCall/split_inputs/split/_2]]
(1) Invalid argument: Number of ways to split should evenly divide the split dimension, but got split_dim 0 (size = 1) and num_split 2
[[node PartitionedCall/split_inputs/split (defined at image_classification.py:134) ]]
0 successful operations.
1 derived errors ignored. [Op:__inference_pruned_25839]
Function call stack:
pruned -> pruned
Further testing points me to this line in the log where the TRT Engine is built when trying to run with even batch sizes:
Building a new TensorRT engine for PartitionedCall/TRTEngineOp_0 with input shapes: [[1,224,224,3], [1,224,224,3]]
How can we modify this to support odd batch sizes as well?
@pooyadavoodi @aaroey Any ideas for above?
use the nvidia tensorflow container nvcr.io/nvidia/tensorflow:20.01-tf2-py3. this is working fine for image classification models although object detection is segmentation fault for NMS.
@tfeher @bixia1 could you help to take a look?
Hi everyone, I'm trying to run the Image Classification example with TF 2.0 and TensorRT 7. The changes in the API have now made it that the dependency on NETS has been removed but now, it seems that the code asks for a frozen graph file. I tried downloading the file from here: http://download.tensorflow.org/models/official/resnet_v1_imagenet_savedmodel.tar.gz and then I put it in a folder and gave the following arguments for a synthetic run:
python image_classification.py --input_saved_model_dir data/resnet_v1_50/1523293981 --output_saved_model_dir trt_engine --batch_size 128 --data_dir . --use_trt --num_iterations 100 --use_synthetic --precision FP16 --mode benchmark
The output that I got was this:
Can anyone explain the error or what I should be doing instead?