search

tensorflow / tensorrt

TensorFlow/TensorRT integration
Apache License 2.0
734 stars 223 forks source link

inceptionv3 c++ example inference build trt engine failed； #331

Open Weili17 opened 1 year ago

Weili17 commented 1 year ago

environment：docker nvcr.io/nvidia/tensorflow:22.06-tf2-py3 Reason: TF-TRT Warning: Engine creation for PartitionedCall/PartitionedCall/TRTEngineOp_000_000 failed. The native segment will be used instead. Reason: NOT_FOUND: No converter for op _FusedBatchNormEx

Weili17 commented 1 year ago

environment：docker nvcr.io/nvidia/tensorflow:22.06-tf2-py3 log： 2023-03-02 05:03:37.809625: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable TF_ENABLE_ONEDNN_OPTS=0. 2023-03-02 05:03:37.934203: I tensorflow/core/platform/cpu_feature_guard.cc:194] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: SSE3 SSE4.1 SSE4.2 AVX To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-03-02 05:03:39.032139: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 11377 MB memory: -> device: 0, name: Tesla T4, pci bus id: 0000:af:00.0, compute capability: 7.5 2023-03-02 05:03:39.032737: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 13807 MB memory: -> device: 1, name: Tesla T4, pci bus id: 0000:d8:00.0, compute capability: 7.5 2023-03-02 05:03:39.278224: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 11377 MB memory: -> device: 0, name: Tesla T4, pci bus id: 0000:af:00.0, compute capability: 7.5 2023-03-02 05:03:39.278473: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 13807 MB memory: -> device: 1, name: Tesla T4, pci bus id: 0000:d8:00.0, compute capability: 7.5 2023-03-02 05:03:42.943295: I tensorflow/compiler/tf2tensorrt/common/utils.cc:104] Linked TensorRT version: 8.2.5 2023-03-02 05:03:42.984223: I tensorflow/compiler/tf2tensorrt/common/utils.cc:106] Loaded TensorRT version: 8.2.5 2023-03-02 05:03:44.790395: W tensorflow/compiler/tf2tensorrt/kernels/trt_engine_op.cc:1055] TF-TRT Warning: Engine creation for PartitionedCall/PartitionedCall/TRTEngineOp_000_000 failed. The native segment will be used instead. Reason: NOT_FOUND: No converter for op _FusedBatchNormEx 2023-03-02 05:03:44.790482: W tensorflow/compiler/tf2tensorrt/kernels/trt_engine_op.cc:888] TF-TRT Warning: Engine retrieval for input shapes: [[1,224,224,3]] failed. Running native segment for PartitionedCall/PartitionedCall/TRTEngineOp_000_000 2023-03-02 05:03:46.008720: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8401 2023-03-02 05:03:47.439435: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 11377 MB memory: -> device: 0, name: Tesla T4, pci bus id: 0000:af:00.0, compute capability: 7.5 2023-03-02 05:03:47.439682: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 13807 MB memory: -> device: 1, name: Tesla T4, pci bus id: 0000:d8:00.0, compute capability: 7.5 2023-03-02 05:03:47.444964: I tensorflow/examples/image_classification/main.cc:276] malamute (250): 0.575496 2023-03-02 05:03:47.444990: I tensorflow/examples/image_classification/main.cc:276] Saint Bernard (248): 0.399285 2023-03-02 05:03:47.444998: I tensorflow/examples/image_classification/main.cc:276] Eskimo dog (249): 0.0228339 2023-03-02 05:03:47.445006: I tensorflow/examples/image_classification/main.cc:276] Ibizan hound (174): 0.00127912 2023-03-02 05:03:47.445013: I tensorflow/examples/image_classification/main.cc:276] Mexican hairless (269): 0.000520922 2023-03-02 05:03:47.445205: W tensorflow/compiler/tf2tensorrt/kernels/trt_engine_op.cc:888] TF-TRT Warning: Engine retrieval for input shapes: [[1,224,224,3]] failed. Running native segment for PartitionedCall/PartitionedCall/TRTEngineOp_000_000 2023-03-02 05:03:47.538143: W tensorflow/compiler/tf2tensorrt/kernels/trt_engine_op.cc:888] TF-TRT Warning: Engine retrieval for input shapes: [[1,224,224,3]] failed. Running native segment for PartitionedCall/PartitionedCall/TRTEngineOp_000_000 2023-03-02 05:03:47.553149: W tensorflow/compiler/tf2tensorrt/kernels/trt_engine_op.cc:888] TF-TRT Warning: Engine retrieval for input shapes: [[1,224,224,3]] failed. Running native segment for PartitionedCall/PartitionedCall/TRTEngineOp_000_000 2023-03-02 05:03:47.558721: W tensorflow/compiler/tf2tensorrt/kernels/trt_engine_op.cc:888] TF-TRT Warning: Engine retrieval for input shapes: [[1,224,224,3]] failed. Running native segment for PartitionedCall/PartitionedCall/TRTEngineOp_000_000 2023-03-02 05:03:47.857838: I tensorflow/examples/image_classification/main.cc:423] Step 0: 6 ms 2023-03-02 05:03:48.168601: I tensorflow/examples/image_classification/main.cc:423] Step 50: 6 ms 2023-03-02 05:03:48.492328: I tensorflow/examples/image_classification/main.cc:423] Step 100: 6.0198 ms 2023-03-02 05:03:48.805529: I tensorflow/examples/image_classification/main.cc:423] Step 150: 6.01325 ms 2023-03-02 05:03:49.118205: I tensorflow/examples/image_classification/main.cc:423] Step 200: 6.00995 ms 2023-03-02 05:03:49.436369: I tensorflow/examples/image_classification/main.cc:423] Step 250: 6.01195 ms 2023-03-02 05:03:49.751933: I tensorflow/examples/image_classification/main.cc:423] Step 300: 6.01329 ms 2023-03-02 05:03:50.065457: I tensorflow/examples/image_classification/main.cc:423] Step 350: 6.0114 ms 2023-03-02 05:03:50.381459: I tensorflow/examples/image_classification/main.cc:423] Step 400: 6.01247 ms 2023-03-02 05:03:50.698780: I tensorflow/examples/image_classification/main.cc:423] Step 450: 6.01109 ms 2023-03-02 05:03:51.011541: I tensorflow/examples/image_classification/main.cc:423] Step 500: 6.00998 ms 2023-03-02 05:03:51.327086: I tensorflow/examples/image_classification/main.cc:423] Step 550: 6.00907 ms 2023-03-02 05:03:51.643465: I tensorflow/examples/image_classification/main.cc:423] Step 600: 6.00832 ms 2023-03-02 05:03:51.961735: I tensorflow/examples/image_classification/main.cc:423] Step 650: 6.01229 ms 2023-03-02 05:03:52.281288: I tensorflow/examples/image_classification/main.cc:423] Step 700: 6.01712 ms 2023-03-02 05:03:52.601280: I tensorflow/examples/image_classification/main.cc:423] Step 750: 6.01598 ms 2023-03-02 05:03:52.922119: I tensorflow/examples/image_classification/main.cc:423] Step 800: 6.01998 ms 2023-03-02 05:03:53.240485: I tensorflow/examples/image_classification/main.cc:423] Step 850: 6.01998 ms 2023-03-02 05:03:53.558849: I tensorflow/examples/image_classification/main.cc:423] Step 900: 6.01887 ms 2023-03-02 05:03:53.875391: I tensorflow/examples/image_classification/main.cc:423] Step 950: 6.01788 ms 2023-03-02 05:03:54.186272: I tensorflow/examples/image_classification/main.cc:426] Throughput: 166.196 images/s