Closed youtalk closed 1 year ago
I was trying the Quickstart: https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_proximity_segmentation#quickstart with Jetson Orin Dev Kit with JetPack 5.1.1 but the trtexec failed. Do you have any idea? The following log described Cannot create DLA engine, 0 not available. I think it is a docker device mount problem.
trtexec
Cannot create DLA engine, 0 not available
/usr/src/tensorrt/bin/trtexec --saveEngine=/tmp/models/bi3d/bi3dnet_featnet.plan \ > --onnx=/tmp/models/bi3d/featnet.onnx \ > --int8 --useDLACore=0 --allowGPUFallback && > /usr/src/tensorrt/bin/trtexec --saveEngine=/tmp/models/bi3d/bi3dnet_segnet.plan \ > --onnx=/tmp/models/bi3d/segnet.onnx \ > --int8 --useDLACore=0 --allowGPUFallback &&&& RUNNING TensorRT.trtexec [TensorRT v8502] # /usr/src/tensorrt/bin/trtexec --saveEngine=/tmp/models/bi3d/bi3dnet_featnet.plan --onnx=/tmp/models/bi3d/featnet.onnx --int8 --useDLACore=0 --allowGPUFallback [04/22/2023-09:32:31] [I] === Model Options === [04/22/2023-09:32:31] [I] Format: ONNX [04/22/2023-09:32:31] [I] Model: /tmp/models/bi3d/featnet.onnx [04/22/2023-09:32:31] [I] Output: [04/22/2023-09:32:31] [I] === Build Options === [04/22/2023-09:32:31] [I] Max batch: explicit batch [04/22/2023-09:32:31] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default [04/22/2023-09:32:31] [I] minTiming: 1 [04/22/2023-09:32:31] [I] avgTiming: 8 [04/22/2023-09:32:31] [I] Precision: FP32+INT8 [04/22/2023-09:32:31] [I] LayerPrecisions: [04/22/2023-09:32:31] [I] Calibration: Dynamic [04/22/2023-09:32:31] [I] Refit: Disabled [04/22/2023-09:32:31] [I] Sparsity: Disabled [04/22/2023-09:32:31] [I] Safe mode: Disabled [04/22/2023-09:32:31] [I] DirectIO mode: Disabled [04/22/2023-09:32:31] [I] Restricted mode: Disabled [04/22/2023-09:32:31] [I] Build only: Disabled [04/22/2023-09:32:31] [I] Save engine: /tmp/models/bi3d/bi3dnet_featnet.plan [04/22/2023-09:32:31] [I] Load engine: [04/22/2023-09:32:31] [I] Profiling verbosity: 0 [04/22/2023-09:32:31] [I] Tactic sources: Using default tactic sources [04/22/2023-09:32:31] [I] timingCacheMode: local [04/22/2023-09:32:31] [I] timingCacheFile: [04/22/2023-09:32:31] [I] Heuristic: Disabled [04/22/2023-09:32:31] [I] Preview Features: Use default preview flags. [04/22/2023-09:32:31] [I] Input(s)s format: fp32:CHW [04/22/2023-09:32:31] [I] Output(s)s format: fp32:CHW [04/22/2023-09:32:31] [I] Input build shapes: model [04/22/2023-09:32:31] [I] Input calibration shapes: model [04/22/2023-09:32:31] [I] === System Options === [04/22/2023-09:32:31] [I] Device: 0 [04/22/2023-09:32:31] [I] DLACore: 0(With GPU fallback) [04/22/2023-09:32:31] [I] Plugins: [04/22/2023-09:32:31] [I] === Inference Options === [04/22/2023-09:32:31] [I] Batch: Explicit [04/22/2023-09:32:31] [I] Input inference shapes: model [04/22/2023-09:32:31] [I] Iterations: 10 [04/22/2023-09:32:31] [I] Duration: 3s (+ 200ms warm up) [04/22/2023-09:32:31] [I] Sleep time: 0ms [04/22/2023-09:32:31] [I] Idle time: 0ms [04/22/2023-09:32:31] [I] Streams: 1 [04/22/2023-09:32:31] [I] ExposeDMA: Disabled [04/22/2023-09:32:31] [I] Data transfers: Enabled [04/22/2023-09:32:31] [I] Spin-wait: Disabled [04/22/2023-09:32:31] [I] Multithreading: Disabled [04/22/2023-09:32:31] [I] CUDA Graph: Disabled [04/22/2023-09:32:31] [I] Separate profiling: Disabled [04/22/2023-09:32:31] [I] Time Deserialize: Disabled [04/22/2023-09:32:31] [I] Time Refit: Disabled [04/22/2023-09:32:31] [I] NVTX verbosity: 0 [04/22/2023-09:32:31] [I] Persistent Cache Ratio: 0 [04/22/2023-09:32:31] [I] Inputs: [04/22/2023-09:32:31] [I] === Reporting Options === [04/22/2023-09:32:31] [I] Verbose: Disabled [04/22/2023-09:32:31] [I] Averages: 10 inferences [04/22/2023-09:32:31] [I] Percentiles: 90,95,99 [04/22/2023-09:32:31] [I] Dump refittable layers:Disabled [04/22/2023-09:32:31] [I] Dump output: Disabled [04/22/2023-09:32:31] [I] Profile: Disabled [04/22/2023-09:32:31] [I] Export timing to JSON file: [04/22/2023-09:32:31] [I] Export output to JSON file: [04/22/2023-09:32:31] [I] Export profile to JSON file: [04/22/2023-09:32:31] [I] [04/22/2023-09:32:31] [I] === Device Information === [04/22/2023-09:32:31] [I] Selected Device: Orin [04/22/2023-09:32:31] [I] Compute Capability: 8.7 [04/22/2023-09:32:31] [I] SMs: 8 [04/22/2023-09:32:31] [I] Compute Clock Rate: 0.624 GHz [04/22/2023-09:32:31] [I] Device Global Memory: 6480 MiB [04/22/2023-09:32:31] [I] Shared Memory per SM: 164 KiB [04/22/2023-09:32:31] [I] Memory Bus Width: 64 bits (ECC disabled) [04/22/2023-09:32:31] [I] Memory Clock Rate: 0.624 GHz [04/22/2023-09:32:31] [I] [04/22/2023-09:32:31] [I] TensorRT version: 8.5.2 [04/22/2023-09:32:32] [I] [TRT] [MemUsageChange] Init CUDA: CPU +220, GPU +0, now: CPU 249, GPU 4170 (MiB) [04/22/2023-09:32:35] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +302, GPU +283, now: CPU 574, GPU 4472 (MiB) [04/22/2023-09:32:35] [I] Start parsing network model [04/22/2023-09:32:35] [I] [TRT] ---------------------------------------------------------------- [04/22/2023-09:32:35] [I] [TRT] Input filename: /tmp/models/bi3d/featnet.onnx [04/22/2023-09:32:35] [I] [TRT] ONNX IR version: 0.0.7 [04/22/2023-09:32:35] [I] [TRT] Opset version: 13 [04/22/2023-09:32:35] [I] [TRT] Producer name: pytorch [04/22/2023-09:32:35] [I] [TRT] Producer version: 1.10 [04/22/2023-09:32:35] [I] [TRT] Domain: [04/22/2023-09:32:35] [I] [TRT] Model version: 0 [04/22/2023-09:32:35] [I] [TRT] Doc string: [04/22/2023-09:32:35] [I] [TRT] ---------------------------------------------------------------- [04/22/2023-09:32:35] [I] Finish parsing network model [04/22/2023-09:32:35] [I] FP32 and INT8 precisions have been specified - more performance might be enabled by additionally specifying --fp16 or --best [04/22/2023-09:32:35] [E] Cannot create DLA engine, 0 not available [04/22/2023-09:32:35] [E] Network And Config setup failed [04/22/2023-09:32:35] [E] Building engine failed [04/22/2023-09:32:35] [E] Failed to create engine from model or file. [04/22/2023-09:32:35] [E] Engine set up failed &&&& FAILED TensorRT.trtexec [TensorRT v8502] # /usr/src/tensorrt/bin/trtexec --saveEngine=/tmp/models/bi3d/bi3dnet_featnet.plan --onnx=/tmp/models/bi3d/featnet.onnx --int8 --useDLACore=0 --allowGPUFallback
I have encountered this problem too. Would you mind talking about how you fix this problem, please? Thank you!
I was trying the Quickstart: https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_proximity_segmentation#quickstart with Jetson Orin Dev Kit with JetPack 5.1.1 but the
trtexec
failed. Do you have any idea? The following log describedCannot create DLA engine, 0 not available
. I think it is a docker device mount problem.