Jetson AGX Orin package version

[asd841018]

Hi, thanks for awesome project !

I am curious about what's your Jetson AGX Orin package version, like jetpack, cuda, cudnn and tensorrt version.

Cause I have the device but I could not transfer the l0 onnx model to tensorrt engine.

By the way, I am using jetpack 8.6.

[zqd-big]

嘿，请问你的问题解决了吗，我也在试着在jetson orin nx设备上转tensorrt，结果失败，我的jetpack版本为5.1.1，报错： /Add, /image_encoder/norm/Sqrt), /image_encoder/norm/Div) [02/27/2024-09:08:48] [I] [TRT] [GpuLayer] SCALE: onnx::Mul_1473 + /image_encoder/norm/Mul_1 + onnx::Add_1474 + /image_encoder/norm/Add_1 [02/27/2024-09:08:49] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +534, GPU +409, now: CPU 1230, GPU 5605 (MiB) [02/27/2024-09:08:49] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +83, GPU +79, now: CPU 1313, GPU 5684 (MiB) [02/27/2024-09:08:49] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored. [02/27/2024-09:15:57] [W] [TRT] Tactic Device request: 13089MB Available: 9185MB. Device memory is insufficient to use tactic. [02/27/2024-09:15:57] [W] [TRT] Skipping tactic 3 due to insufficient memory on requested size of 13089 detected for tactic 0x0000000000000004. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/27/2024-09:15:58] [W] [TRT] Tactic Device request: 13089MB Available: 9185MB. Device memory is insufficient to use tactic. [02/27/2024-09:15:58] [W] [TRT] Skipping tactic 8 due to insufficient memory on requested size of 13089 detected for tactic 0x000000000000003c. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/27/2024-09:15:58] [W] [TRT] Tactic Device request: 13089MB Available: 9185MB. Device memory is insufficient to use tactic. [02/27/2024-09:15:59] [W] [TRT] Skipping tactic 13 due to insufficient memory on requested size of 13089 detected for tactic 0x0000000000000074. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/27/2024-09:21:33] [I] [TRT] [GraphReduction] The approximate region cut reduction algorithm is called. [02/27/2024-09:21:33] [I] [TRT] Total Activation Memory: 16794314752 [02/27/2024-09:21:33] [I] [TRT] Detected 1 inputs and 1 output network tensors. [02/27/2024-09:21:34] [I] [TRT] Total Host Persistent Memory: 238528 [02/27/2024-09:21:34] [I] [TRT] Total Device Persistent Memory: 0 [02/27/2024-09:21:34] [I] [TRT] Total Scratch Memory: 2048 [02/27/2024-09:21:34] [I] [TRT] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 12 MiB, GPU 8491 MiB [02/27/2024-09:21:34] [I] [TRT] [BlockAssignment] Started assigning block shifts. This will take 221 steps to complete. [02/27/2024-09:21:34] [I] [TRT] [BlockAssignment] Algorithm ShiftNTopDown took 29.4971ms to assign 7 blocks to 221 nodes requiring 80740352 bytes. [02/27/2024-09:21:34] [I] [TRT] Total Activation Memory: 80740352 [02/27/2024-09:21:34] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +0, GPU +0, now: CPU 1641, GPU 6839 (MiB) [02/27/2024-09:21:34] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +0, GPU +128, now: CPU 0, GPU 128 (MiB) [02/27/2024-09:21:34] [E] Saving engine to file failed. [02/27/2024-09:21:34] [E] Engine set up failed &&&& FAILED TensorRT.trtexec [TensorRT v8502] # trtexec --onnx=/data/l0_encoder.onnx --minShapes=input_image:1x3x512x512 --optShapes=input_image:1x3x512x512 --maxShapes=input_image:1x3x512x512 --saveEngine=~/l0_encoder.engine

[zhuoyang20]

Hi asd841018 and zqd-big,

TensorRT tries various optimization tactics during the build phase. It looks like there is a tactic that tries to use more memory than the Jetson AGX Orin device has. You can restrict the workspace size in trtexec.

--workspace=N   #Set workspace size in MiB.

Best, Zhuoyang