benchmark error - Githubissues

AlexiAlp commented 1 week ago

Search before asking

[x] I have searched the Ultralytics YOLO issues and found no similar bug report.

Ultralytics YOLO Component

No response

Bug

The scripts i wrote is

from ultralytics.utils.benchmarks import benchmark benchmark(model="yolo11n.pt", data="coco8.yaml", imgsz=640, format="onnx")

(the same is https://docs.ultralytics.com/zh/modes/benchmark/#usage-examples):, but i got the error: ONNX: starting export with onnx 1.17.0 opset 19... ONNX: slimming with onnxslim 0.1.48... ONNX: export success ✅ 0.7s, saved as 'yolo11n.onnx' (10.2 MB)

Export complete (0.9s) Results saved to /data/code/yolo11/ultralytics Predict: yolo predict task=detect model=yolo11n.onnx imgsz=640
Validate: yolo val task=detect model=yolo11n.onnx imgsz=640 data=/usr/src/ultralytics/ultralytics/cfg/datasets/coco.yaml
Visualize: https://netron.app ERROR ❌️ Benchmark failure for ONNX: model='/data/code/yolo11/ultralytics/ultralytics/cfg/datasets/coco.yaml' is not a supported model format. Ultralytics supports: ('PyTorch', 'TorchScript', 'ONNX', 'OpenVINO', 'TensorRT', 'CoreML', 'TensorFlow SavedModel', 'TensorFlow GraphDef', 'TensorFlow Lite', 'TensorFlow Edge TPU', 'TensorFlow.js', 'PaddlePaddle', 'MNN', 'NCNN', 'IMX', 'RKNN') See https://docs.ultralytics.com/modes/predict for help. Setup complete ✅ (8 CPUs, 61.4 GB RAM, 461.1/499.8 GB disk) Traceback (most recent call last): File "/data/code/yolo11/ultralytics/onnx_run.py", line 7, in benchmark(model="yolo11n.pt", data="coco8.yaml", imgsz=640, format="onnx") File "/data/code/yolo11/ultralytics/ultralytics/utils/benchmarks.py", line 183, in benchmark df = pd.DataFrame(y, columns=["Format", "Status❔", "Size (MB)", key, "Inference time (ms/im)", "FPS"]) UnboundLocalError: local variable 'key' referenced before assignment

Environment

Ultralytics 8.3.70 🚀 Python-3.10.0 torch-2.6.0+cu124 CUDA:0 (NVIDIA L20, 45589MiB) Setup complete ✅ (8 CPUs, 61.4 GB RAM, 461.2/499.8 GB disk)

OS Linux-5.15.0-124-generic-x86_64-with-glibc2.35 Environment Linux Python 3.10.0 Install pip RAM 61.43 GB Disk 461.2/499.8 GB CPU Intel Xeon Gold 6462C CPU count 8 GPU NVIDIA L20, 45589MiB GPU count 1 CUDA 12.4

numpy ✅ 2.1.1<=2.1.1,>=1.23.0 matplotlib ✅ 3.10.0>=3.3.0 opencv-python ✅ 4.11.0.86>=4.6.0 pillow ✅ 11.1.0>=7.1.2 pyyaml ✅ 6.0.2>=5.3.1 requests ✅ 2.32.3>=2.23.0 scipy ✅ 1.15.1>=1.4.1 torch ✅ 2.6.0>=1.8.0 torch ✅ 2.6.0!=2.4.0,>=1.8.0; sys_platform == "win32" torchvision ✅ 0.21.0>=0.9.0 tqdm ✅ 4.67.1>=4.64.0 psutil ✅ 6.1.1 py-cpuinfo ✅ 9.0.0 pandas ✅ 2.2.3>=1.1.4 seaborn ✅ 0.13.2>=0.11.0 ultralytics-thop ✅ 2.0.14>=2.0.0

Minimal Reproducible Example

from ultralytics.utils.benchmarks import benchmark benchmark(model="yolo11n.pt", data="coco8.yaml", imgsz=640, format="onnx")

Additional

No response

Are you willing to submit a PR?

[ ] Yes I'd like to help by submitting a PR!

UltralyticsAssistant commented 1 week ago

👋 Hello @AlexiAlp, thank you for your detailed report and for your interest in Ultralytics 🚀! We recommend starting with the Ultralytics Docs to explore all functionality, including Modes and Usage Examples.

Bug Report

It looks like you've encountered a 🐛 bug. To help us debug this issue effectively, could you please confirm if the provided code snippet is the Minimum Reproducible Example (MRE)? If not, kindly update it with an isolated example that reproduces the error. Learn more about MREs here.

Additionally:

Double-check that you're using the latest version of ultralytics. Try upgrading with the following command:
```
pip install -U ultralytics
```
Ensure all dependencies such as PyTorch and ONNX are up to date.

Real-Time Support

If you'd like input from the community while waiting for Ultralytics support:

Join our Discord for real-time discussions 🎧.
Engage in topic-based discussions on our Discourse Community.
Explore additional insights or share your scenario on our Subreddit.

Environments

YOLO models work seamlessly across the environments listed below. You may want to test your scenario in one of them to rule out environment-specific issues:

Notebooks with free GPU:
Google Cloud Deep Learning VM. See GCP Quickstart Guide
Amazon Deep Learning AMI. See AWS Quickstart Guide
Docker Image. See Docker Quickstart Guide

For further options related to model formats and troubleshooting, please visit our Prediction Documentation.

Status and Updates

You can check the current CI status of the repository here:

This is an automated response to ensure swift engagement. An Ultralytics engineer will review and assist with your issue as soon as possible 😊. Thank you for your patience!

Y-T-G commented 1 week ago

You have an ultralytics folder which is outdated or broken in the same directory as the script. Delete the folder, or move your script to a folder where there's no ultralytics folder.

AlexiAlp commented 1 week ago

ok, I will try, it works when i remove the project code folder and reinstall the env! But i encounter an error about benchmark onnx int8 error: benchmark(model="./yolo11n.pt", data="./coco.yaml", imgsz=640, int8=True, device="cuda", format="onnx") When i set int8=False, the progrom work well, but i set int8=True, it comes the following error:

ERROR ❌️ Benchmark failure for ONNX: ERROR ❌️ argument 'int8' is not supported for format='onnx' Setup complete ✅ (8 CPUs, 61.4 GB RAM, 468.6/499.8 GB disk) Traceback (most recent call last): File "/data/code/yolo11/onnx_benchmark.py", line 4, in benchmark(model="./yolo11n.pt", data="./coco.yaml", imgsz=640, int8=True, device="cuda", format="onnx") File "/root/miniconda3/envs/yolo11_test/lib/python3.10/site-packages/ultralytics/utils/benchmarks.py", line 183, in benchmark df = pd.DataFrame(y, columns=["Format", "Status❔", "Size (MB)", key, "Inference time (ms/im)", "FPS"]) UnboundLocalError: local variable 'key' referenced before assignment

Y-T-G commented 1 week ago

int8 is not supported for ONNX like it says

AlexiAlp commented 1 week ago

OK

glenn-jocher commented 1 week ago

The int8 quantization is not supported for ONNX exports in Ultralytics. This parameter is only applicable to formats like TensorRT, OpenVINO, and CoreML as shown in our export formats table. For ONNX benchmarking, please omit the int8=True argument. If you require INT8 quantization, consider using TensorRT format which supports it natively - see our TensorRT INT8 guide.

iokarkan commented 5 days ago

Edited to include complete output of benchmark command.

I am facing the same problem with a Jetson Orin Nano 8GB and the official docker image as described in the Jetson docs.

I start the container of ultralytics 8.3.51 with

t=ultralytics/ultralytics:8.3.51-jetson-jetpack5
sudo docker pull $t && sudo docker run -it --ipc=host --runtime=nvidia $t

I verify the version 8.3.51 running:

ultralytics version
# returns 8.3.51

For completeness, running yolo checks returns:

WARNING ⚠️ torchvision==0.14 is incompatible with torch==2.0.
Run 'pip install torchvision==0.15' to fix torchvision or 'pip install -U torch torchvision' to update both.
For a full compatibility table see https://github.com/pytorch/vision#installation
Creating new Ultralytics Settings v0.0.6 file ✅
View Ultralytics Settings with 'yolo settings' or at '/root/.config/Ultralytics/settings.json'
Update Settings with 'yolo settings key=value', i.e. 'yolo settings runs_dir=path/to/dir'. For help see https://docs.ultralytics.com/quickstart/#ultralytics-settings.
Ultralytics 8.3.51 🚀 Python-3.8.10 torch-2.0.0a0+ec3941ad.nv23.02 CUDA:0 (Orin, 7337MiB)
Setup complete ✅ (6 CPUs, 7.2 GB RAM, 134.5/233.7 GB disk)

OS                  Linux-5.10.104-tegra-aarch64-with-glibc2.29
Environment         Docker
Python              3.8.10
Install             git
RAM                 7.16 GB
Disk                134.5/233.7 GB
CPU                 ARMv8 Processor rev 1 (v8l)
CPU count           6
GPU                 Orin, 7337MiB
GPU count           1
CUDA                11.4

numpy               ✅ 1.23.5>=1.23.0
matplotlib          ✅ 3.7.5>=3.3.0
pillow              ✅ 10.4.0>=7.1.2
pyyaml              ✅ 6.0.2>=5.3.1
requests            ✅ 2.32.3>=2.23.0
scipy               ✅ 1.10.1>=1.4.1
torch               ✅ 2.0.0a0+ec3941ad.nv23.2>=1.8.0
torchvision         ✅ 0.14.1>=0.9.0
tqdm                ✅ 4.67.1>=4.64.0
psutil              ✅ 6.1.0
py-cpuinfo          ✅ 9.0.0
pandas              ✅ 2.0.3>=1.1.4
seaborn             ✅ 0.13.2>=0.11.0
ultralytics-thop    ✅ 2.0.13>=2.0.0
numpy               ✅ 1.23.5<2.0.0; sys_platform == "darwin"
torch               ✅ 2.0.0a0+ec3941ad.nv23.2!=2.4.0,>=1.8.0; sys_platform == "win32"

If I then run benchmarks: yolo benchmark model=yolo11n.pt data=coco8.yaml imgsz=640 (full output below)

WARNING ⚠️ torchvision==0.14 is incompatible with torch==2.0. Run 'pip install torchvision==0.15' to fix torchvision or 'pip install -U torch torchvision' to update both. For a full compatibility table see https://github.com/pytorch/vision#installation /usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: warn(f"Failed to load image Python extension: {e}") ERROR ❌️ Benchmark failure for PyTorch: Couldn't load custom C++ ops. This can happen if your PyTorch and torchvision versions are incompatible, or if you had errors while compiling torchvision from source. For further information on the compatible versions, check https://github.com/pytorch/vision#installation for the compatibility matrix. Please check your PyTorch version with torch.__version__ and your torchvision version with torchvision.__version__ and verify if they are compatible, and if not please reinstall torchvision so that it matches your PyTorch install. PyTorch: starting from 'yolo11n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (5.4 MB) TorchScript: starting export with torch 2.0.0a0+ec3941ad.nv23.02... TorchScript: export success ✅ 2.7s, saved as 'yolo11n.torchscript' (10.5 MB) Export complete (2.8s) Results saved to /ultralytics Predict: yolo predict task=detect model=yolo11n.torchscript imgsz=640 Validate: yolo val task=detect model=yolo11n.torchscript imgsz=640 data=/usr/src/ultralytics/ultralytics/cfg/datasets/coco.yaml Visualize: https://netron.app Loading yolo11n.torchscript for TorchScript inference... ERROR ❌️ Benchmark failure for TorchScript: Couldn't load custom C++ ops. This can happen if your PyTorch and torchvision versions are incompatible, or if you had errors while compiling torchvision from source. For further information on the compatible versions, check https://github.com/pytorch/vision#installation for the compatibility matrix. Please check your PyTorch version with torch.__version__ and your torchvision version with torchvision.__version__ and verify if they are compatible, and if not please reinstall torchvision so that it matches your PyTorch install. PyTorch: starting from 'yolo11n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (5.4 MB) requirements: Ultralytics requirement ['onnxslim'] not found, attempting AutoUpdate... WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.3.1 -> 25.0 [notice] To update, run: python3 -m pip install --upgrade pip Collecting onnxslim Downloading onnxslim-0.1.48-py3-none-any.whl.metadata (4.6 kB) Requirement already satisfied: onnx in /usr/local/lib/python3.8/dist-packages (from onnxslim) (1.17.0) Requirement already satisfied: sympy in /usr/local/lib/python3.8/dist-packages (from onnxslim) (1.11.1) Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from onnxslim) (20.9) Requirement already satisfied: numpy>=1.20 in /usr/local/lib/python3.8/dist-packages (from onnx->onnxslim) (1.23.5) Requirement already satisfied: protobuf>=3.20.2 in /usr/local/lib/python3.8/dist-packages (from onnx->onnxslim) (4.25.5) Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->onnxslim) (3.1.4) Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.8/dist-packages (from sympy->onnxslim) (1.2.1) Downloading onnxslim-0.1.48-py3-none-any.whl (142 kB) Installing collected packages: onnxslim Successfully installed onnxslim-0.1.48 requirements: AutoUpdate success ✅ 2.5s, installed 1 package: ['onnxslim'] requirements: ⚠️ Restart runtime or rerun command for updates to take effect ONNX: starting export with onnx 1.17.0 opset 16... ====== Diagnostic Run torch.onnx.export version 2.0.0a0+ec3941ad.nv23.02 ======= verbose: False, log level: Level.ERROR ======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ======================== ONNX: slimming with onnxslim 0.1.48... ONNX: export success ✅ 5.5s, saved as 'yolo11n.onnx' (10.2 MB) Export complete (5.7s) Results saved to /ultralytics Predict: yolo predict task=detect model=yolo11n.onnx imgsz=640 Validate: yolo val task=detect model=yolo11n.onnx imgsz=640 data=/usr/src/ultralytics/ultralytics/cfg/datasets/coco.yaml Visualize: https://netron.app Loading yolo11n.onnx for ONNX Runtime inference... Preferring ONNX Runtime TensorrtExecutionProvider ERROR ❌️ Benchmark failure for ONNX: Couldn't load custom C++ ops. This can happen if your PyTorch and torchvision versions are incompatible, or if you had errors while compiling torchvision from source. For further information on the compatible versions, check https://github.com/pytorch/vision#installation for the compatibility matrix. Please check your PyTorch version with torch.__version__ and your torchvision version with torchvision.__version__ and verify if they are compatible, and if not please reinstall torchvision so that it matches your PyTorch install. ERROR ❌️ Benchmark failure for OpenVINO: inference not supported on GPU PyTorch: starting from 'yolo11n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (5.4 MB) ONNX: starting export with onnx 1.17.0 opset 16... ====== Diagnostic Run torch.onnx.export version 2.0.0a0+ec3941ad.nv23.02 ======= verbose: False, log level: Level.ERROR ======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ======================== ONNX: slimming with onnxslim 0.1.48... ONNX: export success ✅ 2.8s, saved as 'yolo11n.onnx' (10.2 MB) TensorRT: starting export with TensorRT 8.5.2.2... [02/05/2025-09:36:14] [TRT] [I] The logger passed into createInferBuilder differs from one already provided for an existing builder, runtime, or refitter. Uses of the global logger, returned by nvinfer1::getLogger(), will return the existing value. [02/05/2025-09:36:14] [TRT] [I] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 2520, GPU 6523 (MiB) [02/05/2025-09:36:14] [TRT] [I] ---------------------------------------------------------------- [02/05/2025-09:36:14] [TRT] [I] Input filename: yolo11n.onnx [02/05/2025-09:36:14] [TRT] [I] ONNX IR version: 0.0.8 [02/05/2025-09:36:14] [TRT] [I] Opset version: 16 [02/05/2025-09:36:14] [TRT] [I] Producer name: pytorch [02/05/2025-09:36:14] [TRT] [I] Producer version: 2.0.0 [02/05/2025-09:36:14] [TRT] [I] Domain: [02/05/2025-09:36:14] [TRT] [I] Model version: 0 [02/05/2025-09:36:14] [TRT] [I] Doc string: [02/05/2025-09:36:14] [TRT] [I] ---------------------------------------------------------------- TensorRT: input "images" with shape(1, 3, 640, 640) DataType.FLOAT TensorRT: output "output0" with shape(1, 84, 8400) DataType.FLOAT TensorRT: building FP32 engine as yolo11n.engine [02/05/2025-09:36:14] [TRT] [I] ---------- Layers Running on DLA ---------- [02/05/2025-09:36:14] [TRT] [I] ---------- Layers Running on GPU ---------- [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.0/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.0/act/Sigmoid), /model.0/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.1/act/Sigmoid), /model.1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.2/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.2/cv1/act/Sigmoid), /model.2/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.2/m.0/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.2/m.0/cv1/act/Sigmoid), /model.2/m.0/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.2/m.0/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(PWN(/model.2/m.0/cv2/act/Sigmoid), /model.2/m.0/cv2/act/Mul), /model.2/m.0/Add) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.2/Split_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.2/Split_output_1 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.2/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.2/cv2/act/Sigmoid), /model.2/cv2/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.3/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.3/act/Sigmoid), /model.3/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.4/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.4/cv1/act/Sigmoid), /model.4/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.4/m.0/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.4/m.0/cv1/act/Sigmoid), /model.4/m.0/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.4/m.0/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(PWN(/model.4/m.0/cv2/act/Sigmoid), /model.4/m.0/cv2/act/Mul), /model.4/m.0/Add) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.4/Split_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.4/Split_output_1 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.4/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.4/cv2/act/Sigmoid), /model.4/cv2/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.5/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.5/act/Sigmoid), /model.5/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.6/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.6/cv1/act/Sigmoid), /model.6/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.6/m.0/cv1/conv/Conv || /model.6/m.0/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.6/m.0/cv1/act/Sigmoid), /model.6/m.0/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.6/m.0/cv2/act/Sigmoid), /model.6/m.0/cv2/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.6/m.0/m/m.0/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.6/m.0/m/m.0/cv1/act/Sigmoid), /model.6/m.0/m/m.0/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.6/m.0/m/m.0/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(PWN(/model.6/m.0/m/m.0/cv2/act/Sigmoid), /model.6/m.0/m/m.0/cv2/act/Mul), /model.6/m.0/m/m.0/Add) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.6/m.0/m/m.1/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.6/m.0/m/m.1/cv1/act/Sigmoid), /model.6/m.0/m/m.1/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.6/m.0/m/m.1/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(PWN(/model.6/m.0/m/m.1/cv2/act/Sigmoid), /model.6/m.0/m/m.1/cv2/act/Mul), /model.6/m.0/m/m.1/Add) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.6/m.0/cv3/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.6/m.0/cv3/act/Sigmoid), /model.6/m.0/cv3/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.6/Split_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.6/Split_output_1 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.6/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.6/cv2/act/Sigmoid), /model.6/cv2/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.7/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.7/act/Sigmoid), /model.7/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.8/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.8/cv1/act/Sigmoid), /model.8/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.8/m.0/cv1/conv/Conv || /model.8/m.0/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.8/m.0/cv1/act/Sigmoid), /model.8/m.0/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.8/m.0/cv2/act/Sigmoid), /model.8/m.0/cv2/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.8/m.0/m/m.0/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.8/m.0/m/m.0/cv1/act/Sigmoid), /model.8/m.0/m/m.0/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.8/m.0/m/m.0/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(PWN(/model.8/m.0/m/m.0/cv2/act/Sigmoid), /model.8/m.0/m/m.0/cv2/act/Mul), /model.8/m.0/m/m.0/Add) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.8/m.0/m/m.1/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.8/m.0/m/m.1/cv1/act/Sigmoid), /model.8/m.0/m/m.1/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.8/m.0/m/m.1/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(PWN(/model.8/m.0/m/m.1/cv2/act/Sigmoid), /model.8/m.0/m/m.1/cv2/act/Mul), /model.8/m.0/m/m.1/Add) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.8/m.0/cv3/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.8/m.0/cv3/act/Sigmoid), /model.8/m.0/cv3/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.8/Split_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.8/Split_output_1 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.8/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.8/cv2/act/Sigmoid), /model.8/cv2/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.9/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.9/cv1/act/Sigmoid), /model.9/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POOLING: /model.9/m/MaxPool [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POOLING: /model.9/m_1/MaxPool [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POOLING: /model.9/m_2/MaxPool [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.9/cv1/act/Mul_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.9/m/MaxPool_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.9/m_1/MaxPool_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.9/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.9/cv2/act/Sigmoid), /model.9/cv2/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.10/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.10/cv1/act/Sigmoid), /model.10/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.10/Split_18 [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.10/m/m.0/attn/qkv/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] SHUFFLE: /model.10/m/m.0/attn/Reshape [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.10/m/m.0/attn/Split [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.10/m/m.0/attn/Split_19 [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.10/m/m.0/attn/Split_20 [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] SHUFFLE: /model.10/m/m.0/attn/Reshape_2 [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] MATRIX_MULTIPLY: /model.10/m/m.0/attn/MatMul [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] SOFTMAX: /model.10/m/m.0/attn/Softmax [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] MATRIX_MULTIPLY: /model.10/m/m.0/attn/MatMul_1 [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] SHUFFLE: /model.10/m/m.0/attn/Reshape_1 [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.10/m/m.0/attn/pe/conv/Conv + /model.10/m/m.0/attn/Add [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.10/m/m.0/attn/proj/conv/Conv + /model.10/m/m.0/Add [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.10/m/m.0/ffn/ffn.0/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.10/m/m.0/ffn/ffn.0/act/Sigmoid), /model.10/m/m.0/ffn/ffn.0/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.10/m/m.0/ffn/ffn.1/conv/Conv + /model.10/m/m.0/Add_1 [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.10/Split_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.10/m/m.0/Add_1_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.10/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.10/cv2/act/Sigmoid), /model.10/cv2/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] RESIZE: /model.11/Resize [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.11/Resize_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.13/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.13/cv1/act/Sigmoid), /model.13/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.13/m.0/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.13/m.0/cv1/act/Sigmoid), /model.13/m.0/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.13/m.0/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(PWN(/model.13/m.0/cv2/act/Sigmoid), /model.13/m.0/cv2/act/Mul), /model.13/m.0/Add) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.13/Split_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.13/Split_output_1 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.13/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.13/cv2/act/Sigmoid), /model.13/cv2/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] RESIZE: /model.14/Resize [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.14/Resize_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.16/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.16/cv1/act/Sigmoid), /model.16/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.16/m.0/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.16/m.0/cv1/act/Sigmoid), /model.16/m.0/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.16/m.0/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(PWN(/model.16/m.0/cv2/act/Sigmoid), /model.16/m.0/cv2/act/Mul), /model.16/m.0/Add) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.16/Split_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.16/Split_output_1 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.16/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.16/cv2/act/Sigmoid), /model.16/cv2/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.17/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv2.0/cv2.0.0/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv3.0/cv3.0.0/cv3.0.0.0/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.17/act/Sigmoid), /model.17/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv2.0/cv2.0.0/act/Sigmoid), /model.23/cv2.0/cv2.0.0/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv3.0/cv3.0.0/cv3.0.0.0/act/Sigmoid), /model.23/cv3.0/cv3.0.0/cv3.0.0.0/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.13/cv2/act/Mul_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv2.0/cv2.0.1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv3.0/cv3.0.0/cv3.0.0.1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.19/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv2.0/cv2.0.1/act/Sigmoid), /model.23/cv2.0/cv2.0.1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv3.0/cv3.0.0/cv3.0.0.1/act/Sigmoid), /model.23/cv3.0/cv3.0.0/cv3.0.0.1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.19/cv1/act/Sigmoid), /model.19/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv2.0/cv2.0.2/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv3.0/cv3.0.1/cv3.0.1.0/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.19/m.0/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv3.0/cv3.0.1/cv3.0.1.0/act/Sigmoid), /model.23/cv3.0/cv3.0.1/cv3.0.1.0/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv3.0/cv3.0.1/cv3.0.1.1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.19/m.0/cv1/act/Sigmoid), /model.19/m.0/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.19/m.0/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv3.0/cv3.0.1/cv3.0.1.1/act/Sigmoid), /model.23/cv3.0/cv3.0.1/cv3.0.1.1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv3.0/cv3.0.2/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(PWN(/model.19/m.0/cv2/act/Sigmoid), /model.19/m.0/cv2/act/Mul), /model.19/m.0/Add) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] SHUFFLE: /model.23/Reshape [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.23/Reshape_copy_output [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.19/Split_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.19/Split_output_1 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.19/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.19/cv2/act/Sigmoid), /model.19/cv2/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.20/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv2.1/cv2.1.0/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv3.1/cv3.1.0/cv3.1.0.0/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.20/act/Sigmoid), /model.20/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv2.1/cv2.1.0/act/Sigmoid), /model.23/cv2.1/cv2.1.0/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv3.1/cv3.1.0/cv3.1.0.0/act/Sigmoid), /model.23/cv3.1/cv3.1.0/cv3.1.0.0/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.10/cv2/act/Mul_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv2.1/cv2.1.1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv3.1/cv3.1.0/cv3.1.0.1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.22/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv2.1/cv2.1.1/act/Sigmoid), /model.23/cv2.1/cv2.1.1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv3.1/cv3.1.0/cv3.1.0.1/act/Sigmoid), /model.23/cv3.1/cv3.1.0/cv3.1.0.1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.22/cv1/act/Sigmoid), /model.22/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv2.1/cv2.1.2/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv3.1/cv3.1.1/cv3.1.1.0/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.22/m.0/cv1/conv/Conv || /model.22/m.0/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv3.1/cv3.1.1/cv3.1.1.0/act/Sigmoid), /model.23/cv3.1/cv3.1.1/cv3.1.1.0/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv3.1/cv3.1.1/cv3.1.1.1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.22/m.0/cv1/act/Sigmoid), /model.22/m.0/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.22/m.0/cv2/act/Sigmoid), /model.22/m.0/cv2/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.22/m.0/m/m.0/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv3.1/cv3.1.1/cv3.1.1.1/act/Sigmoid), /model.23/cv3.1/cv3.1.1/cv3.1.1.1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv3.1/cv3.1.2/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.22/m.0/m/m.0/cv1/act/Sigmoid), /model.22/m.0/m/m.0/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.22/m.0/m/m.0/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] SHUFFLE: /model.23/Reshape_1 [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.23/Reshape_1_copy_output [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(PWN(/model.22/m.0/m/m.0/cv2/act/Sigmoid), /model.22/m.0/m/m.0/cv2/act/Mul), /model.22/m.0/m/m.0/Add) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.22/m.0/m/m.1/cv1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.22/m.0/m/m.1/cv1/act/Sigmoid), /model.22/m.0/m/m.1/cv1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.22/m.0/m/m.1/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(PWN(/model.22/m.0/m/m.1/cv2/act/Sigmoid), /model.22/m.0/m/m.1/cv2/act/Mul), /model.22/m.0/m/m.1/Add) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.22/m.0/cv3/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.22/m.0/cv3/act/Sigmoid), /model.22/m.0/cv3/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.22/Split_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.22/Split_output_1 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.22/cv2/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.22/cv2/act/Sigmoid), /model.22/cv2/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv2.2/cv2.2.0/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv3.2/cv3.2.0/cv3.2.0.0/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv2.2/cv2.2.0/act/Sigmoid), /model.23/cv2.2/cv2.2.0/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv3.2/cv3.2.0/cv3.2.0.0/act/Sigmoid), /model.23/cv3.2/cv3.2.0/cv3.2.0.0/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv2.2/cv2.2.1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv3.2/cv3.2.0/cv3.2.0.1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv2.2/cv2.2.1/act/Sigmoid), /model.23/cv2.2/cv2.2.1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv3.2/cv3.2.0/cv3.2.0.1/act/Sigmoid), /model.23/cv3.2/cv3.2.0/cv3.2.0.1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv2.2/cv2.2.2/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv3.2/cv3.2.1/cv3.2.1.0/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv3.2/cv3.2.1/cv3.2.1.0/act/Sigmoid), /model.23/cv3.2/cv3.2.1/cv3.2.1.0/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv3.2/cv3.2.1/cv3.2.1.1/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(PWN(/model.23/cv3.2/cv3.2.1/cv3.2.1.1/act/Sigmoid), /model.23/cv3.2/cv3.2.1/cv3.2.1.1/act/Mul) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/cv3.2/cv3.2.2/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] SHUFFLE: /model.23/Reshape_2 [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.23/Reshape_2_copy_output [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] SHUFFLE: /model.23/dfl/Reshape + /model.23/dfl/Transpose [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(/model.23/Sigmoid) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] SOFTMAX: /model.23/dfl/Softmax [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONVOLUTION: /model.23/dfl/conv/Conv [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] SHUFFLE: /model.23/dfl/Reshape_1 [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONSTANT: /model.23/Constant_9_output_0 [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] ELEMENTWISE: /model.23/Sub [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONSTANT: /model.23/Constant_10_output_0 [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] ELEMENTWISE: /model.23/Add_1 [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] ELEMENTWISE: /model.23/Sub_1 [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] POINTWISE: PWN(/model.23/Constant_11_output_0 + (Unnamed Layer* 384) [Shuffle], PWN(/model.23/Add_2, /model.23/Div_1)) [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.23/Div_1_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] CONSTANT: /model.23/Constant_12_output_0 + (Unnamed Layer* 388) [Shuffle] [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] ELEMENTWISE: /model.23/Mul_2 [02/05/2025-09:36:14] [TRT] [I] [GpuLayer] COPY: /model.23/Mul_2_output_0 copy [02/05/2025-09:36:14] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 2448, GPU 6435 (MiB) [02/05/2025-09:36:14] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +0, GPU +3, now: CPU 2448, GPU 6438 (MiB) [02/05/2025-09:36:14] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored. [02/05/2025-09:37:10] [TRT] [W] Tactic Device request: 1583MB Available: 1034MB. Device memory is insufficient to use tactic. [02/05/2025-09:37:10] [TRT] [W] Skipping tactic 3 due to insufficient memory on requested size of 1583 detected for tactic 0x0000000000000004. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/05/2025-09:37:10] [TRT] [W] Tactic Device request: 1583MB Available: 1034MB. Device memory is insufficient to use tactic. [02/05/2025-09:37:10] [TRT] [W] Skipping tactic 8 due to insufficient memory on requested size of 1583 detected for tactic 0x000000000000003c. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/05/2025-09:37:10] [TRT] [W] Tactic Device request: 1583MB Available: 1034MB. Device memory is insufficient to use tactic. [02/05/2025-09:37:10] [TRT] [W] Skipping tactic 13 due to insufficient memory on requested size of 1583 detected for tactic 0x0000000000000074. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/05/2025-09:37:18] [TRT] [W] Tactic Device request: 1577MB Available: 1032MB. Device memory is insufficient to use tactic. [02/05/2025-09:37:18] [TRT] [W] Skipping tactic 3 due to insufficient memory on requested size of 1577 detected for tactic 0x0000000000000004. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/05/2025-09:37:18] [TRT] [W] Tactic Device request: 1577MB Available: 1032MB. Device memory is insufficient to use tactic. [02/05/2025-09:37:18] [TRT] [W] Skipping tactic 8 due to insufficient memory on requested size of 1577 detected for tactic 0x000000000000003c. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/05/2025-09:37:18] [TRT] [W] Tactic Device request: 1577MB Available: 1032MB. Device memory is insufficient to use tactic. [02/05/2025-09:37:19] [TRT] [W] Skipping tactic 13 due to insufficient memory on requested size of 1577 detected for tactic 0x0000000000000074. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/05/2025-09:37:42] [TRT] [W] Tactic Device request: 1093MB Available: 959MB. Device memory is insufficient to use tactic. [02/05/2025-09:37:42] [TRT] [W] Skipping tactic 3 due to insufficient memory on requested size of 1093 detected for tactic 0x0000000000000004. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/05/2025-09:37:42] [TRT] [W] Tactic Device request: 1093MB Available: 959MB. Device memory is insufficient to use tactic. [02/05/2025-09:37:42] [TRT] [W] Skipping tactic 8 due to insufficient memory on requested size of 1093 detected for tactic 0x000000000000003c. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/05/2025-09:37:42] [TRT] [W] Tactic Device request: 1093MB Available: 959MB. Device memory is insufficient to use tactic. [02/05/2025-09:37:42] [TRT] [W] Skipping tactic 13 due to insufficient memory on requested size of 1093 detected for tactic 0x0000000000000074. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/05/2025-09:37:47] [TRT] [W] Tactic Device request: 1599MB Available: 958MB. Device memory is insufficient to use tactic. [02/05/2025-09:37:47] [TRT] [W] Skipping tactic 3 due to insufficient memory on requested size of 1599 detected for tactic 0x0000000000000004. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/05/2025-09:37:47] [TRT] [W] Tactic Device request: 1599MB Available: 958MB. Device memory is insufficient to use tactic. [02/05/2025-09:37:47] [TRT] [W] Skipping tactic 8 due to insufficient memory on requested size of 1599 detected for tactic 0x000000000000003c. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/05/2025-09:37:48] [TRT] [W] Tactic Device request: 1599MB Available: 958MB. Device memory is insufficient to use tactic. [02/05/2025-09:37:48] [TRT] [W] Skipping tactic 13 due to insufficient memory on requested size of 1599 detected for tactic 0x0000000000000074. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/05/2025-09:37:51] [TRT] [W] Tactic Device request: 2120MB Available: 957MB. Device memory is insufficient to use tactic. [02/05/2025-09:37:51] [TRT] [W] Skipping tactic 3 due to insufficient memory on requested size of 2120 detected for tactic 0x0000000000000004. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/05/2025-09:37:51] [TRT] [W] Tactic Device request: 2120MB Available: 957MB. Device memory is insufficient to use tactic. [02/05/2025-09:37:51] [TRT] [W] Skipping tactic 8 due to insufficient memory on requested size of 2120 detected for tactic 0x000000000000003c. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/05/2025-09:37:51] [TRT] [W] Tactic Device request: 2120MB Available: 957MB. Device memory is insufficient to use tactic. [02/05/2025-09:37:51] [TRT] [W] Skipping tactic 13 due to insufficient memory on requested size of 2120 detected for tactic 0x0000000000000074. Try decreasing the workspace size with IBuilderConfig::setMemoryPoolLimit(). [02/05/2025-09:38:33] [TRT] [I] Total Activation Memory: 7874038784 [02/05/2025-09:38:33] [TRT] [I] Detected 1 inputs and 3 output network tensors. [02/05/2025-09:38:33] [TRT] [I] Total Host Persistent Memory: 282016 [02/05/2025-09:38:33] [TRT] [I] Total Device Persistent Memory: 0 [02/05/2025-09:38:33] [TRT] [I] Total Scratch Memory: 0 [02/05/2025-09:38:33] [TRT] [I] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 2 MiB, GPU 867 MiB [02/05/2025-09:38:33] [TRT] [I] [BlockAssignment] Started assigning block shifts. This will take 313 steps to complete. [02/05/2025-09:38:33] [TRT] [I] [BlockAssignment] Algorithm ShiftNTopDown took 47.1055ms to assign 10 blocks to 313 nodes requiring 19354624 bytes. [02/05/2025-09:38:33] [TRT] [I] Total Activation Memory: 19354624 [02/05/2025-09:38:33] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +0, GPU +0, now: CPU 2454, GPU 6546 (MiB) [02/05/2025-09:38:33] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +1, GPU +16, now: CPU 1, GPU 51 (MiB) TensorRT: export success ✅ 142.8s, saved as 'yolo11n.engine' (11.8 MB) Export complete (143.0s) Results saved to /ultralytics Predict: yolo predict task=detect model=yolo11n.engine imgsz=640 Validate: yolo val task=detect model=yolo11n.engine imgsz=640 data=/usr/src/ultralytics/ultralytics/cfg/datasets/coco.yaml Visualize: https://netron.app Loading yolo11n.engine for TensorRT inference... [02/05/2025-09:38:33] [TRT] [I] The logger passed into createInferRuntime differs from one already provided for an existing builder, runtime, or refitter. Uses of the global logger, returned by nvinfer1::getLogger(), will return the existing value. [02/05/2025-09:38:33] [TRT] [I] Loaded engine size: 11 MiB [02/05/2025-09:38:34] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +0, GPU +0, now: CPU 2441, GPU 6566 (MiB) [02/05/2025-09:38:34] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +10, now: CPU 0, GPU 45 (MiB) [02/05/2025-09:38:34] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +0, GPU +0, now: CPU 2429, GPU 6566 (MiB) [02/05/2025-09:38:34] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +1, GPU +18, now: CPU 1, GPU 63 (MiB) ERROR ❌️ Benchmark failure for TensorRT: Couldn't load custom C++ ops. This can happen if your PyTorch and torchvision versions are incompatible, or if you had errors while compiling torchvision from source. For further information on the compatible versions, check https://github.com/pytorch/vision#installation for the compatibility matrix. Please check your PyTorch version with torch.__version__ and your torchvision version with torchvision.__version__ and verify if they are compatible, and if not please reinstall torchvision so that it matches your PyTorch install. ERROR ❌️ Benchmark failure for CoreML: inference not supported on GPU PyTorch: starting from 'yolo11n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (5.4 MB) requirements: Ultralytics requirement ['tensorflow-aarch64>=2.0.0'] not found, attempting AutoUpdate... WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.3.1 -> 25.0 [notice] To update, run: python3 -m pip install --upgrade pip Collecting tensorflow-aarch64>=2.0.0 Downloading tensorflow_aarch64-2.13.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (4.0 kB) Requirement already satisfied: absl-py>=1.0.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (2.1.0) Requirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (1.6.3) Requirement already satisfied: flatbuffers>=23.1.21 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (24.3.25) Requirement already satisfied: gast<=0.4.0,>=0.2.1 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (0.4.0) Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (0.2.0) Requirement already satisfied: grpcio<2.0,>=1.24.3 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (1.68.1) Requirement already satisfied: h5py>=2.9.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (3.10.0) Requirement already satisfied: keras<2.14,>=2.13.1 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (2.13.1) Requirement already satisfied: libclang>=13.0.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (18.1.1) Requirement already satisfied: numpy<=1.24.3,>=1.22 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (1.23.5) Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (3.4.0) Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (20.9) Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (4.25.5) Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (75.3.0) Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (1.17.0) Requirement already satisfied: tensorboard<2.14,>=2.13 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (2.13.0) Requirement already satisfied: tensorflow-estimator<2.14,>=2.13.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (2.13.0) Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (2.4.0) Requirement already satisfied: typing-extensions<4.6.0,>=3.6.6 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (4.4.0) Requirement already satisfied: wrapt>=1.11.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (1.17.0) Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /usr/local/lib/python3.8/dist-packages (from tensorflow-aarch64>=2.0.0) (0.35.0) Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/local/lib/python3.8/dist-packages (from astunparse>=1.6.0->tensorflow-aarch64>=2.0.0) (0.45.1) Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.8/dist-packages (from tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (2.37.0) Requirement already satisfied: google-auth-oauthlib<1.1,>=0.5 in /usr/local/lib/python3.8/dist-packages (from tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (1.0.0) Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.8/dist-packages (from tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (3.7) Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.8/dist-packages (from tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (2.32.3) Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /usr/local/lib/python3.8/dist-packages (from tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (0.7.2) Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.8/dist-packages (from tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (3.0.6) Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->tensorflow-aarch64>=2.0.0) (3.1.4) Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.8/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (5.5.0) Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.8/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (0.4.1) Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.8/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (4.9) Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.8/dist-packages (from google-auth-oauthlib<1.1,>=0.5->tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (2.0.0) Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.8/dist-packages (from markdown>=2.6.8->tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (8.5.0) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.8/dist-packages (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (3.4.0) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (3.10) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (2.2.3) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (2024.12.14) Requirement already satisfied: MarkupSafe>=2.1.1 in /usr/local/lib/python3.8/dist-packages (from werkzeug>=1.0.1->tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (2.1.2) Requirement already satisfied: zipp>=3.20 in /usr/local/lib/python3.8/dist-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (3.20.2) Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /usr/local/lib/python3.8/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (0.6.1) Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<1.1,>=0.5->tensorboard<2.14,>=2.13->tensorflow-aarch64>=2.0.0) (3.2.2) Downloading tensorflow_aarch64-2.13.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (248.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 248.7/248.7 MB 17.1 MB/s eta 0:00:00 Installing collected packages: tensorflow-aarch64 Successfully installed tensorflow-aarch64-2.13.1 requirements: AutoUpdate success ✅ 36.1s, installed 1 package: ['tensorflow-aarch64>=2.0.0'] requirements: ⚠️ Restart runtime or rerun command for updates to take effect TensorFlow SavedModel: export failure ❌ 36.4s: /usr/local/lib/python3.8/dist-packages/tensorflow/python/platform/../../../tensorflow_aarch64.libs/libgomp-cc9055c7.so.1.0.0: cannot allocate memory in static TLS block ERROR ❌️ Benchmark failure for TensorFlow SavedModel: /usr/local/lib/python3.8/dist-packages/tensorflow/python/platform/../../../tensorflow_aarch64.libs/libgomp-cc9055c7.so.1.0.0: cannot allocate memory in static TLS block PyTorch: starting from 'yolo11n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (5.4 MB) TensorFlow SavedModel: export failure ❌ 0.0s: /usr/local/lib/python3.8/dist-packages/tensorflow/python/platform/../../../tensorflow_aarch64.libs/libgomp-cc9055c7.so.1.0.0: cannot allocate memory in static TLS block ERROR ❌️ Benchmark failure for TensorFlow GraphDef: /usr/local/lib/python3.8/dist-packages/tensorflow/python/platform/../../../tensorflow_aarch64.libs/libgomp-cc9055c7.so.1.0.0: cannot allocate memory in static TLS block ERROR ❌️ Benchmark failure for TensorFlow Lite: inference not supported on GPU ERROR ❌️ Benchmark failure for TensorFlow Edge TPU: Edge TPU export only supported on non-aarch64 Linux ERROR ❌️ Benchmark failure for TensorFlow.js: inference not supported on GPU PyTorch: starting from 'yolo11n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (5.4 MB) requirements: Ultralytics requirements ['paddlepaddle', 'x2paddle'] not found, attempting AutoUpdate... WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.3.1 -> 25.0 [notice] To update, run: python3 -m pip install --upgrade pip Collecting paddlepaddle Downloading paddlepaddle-2.6.2-cp38-cp38-manylinux2014_aarch64.whl.metadata (8.6 kB) Collecting x2paddle Downloading x2paddle-1.6.0-py3-none-any.whl.metadata (863 bytes) Collecting httpx (from paddlepaddle) Downloading httpx-0.28.1-py3-none-any.whl.metadata (7.1 kB) Requirement already satisfied: numpy>=1.13 in /usr/local/lib/python3.8/dist-packages (from paddlepaddle) (1.23.5) Requirement already satisfied: Pillow in /usr/local/lib/python3.8/dist-packages (from paddlepaddle) (10.4.0) Collecting decorator (from paddlepaddle) Downloading decorator-5.1.1-py3-none-any.whl.metadata (4.0 kB) Collecting astor (from paddlepaddle) Downloading astor-0.8.1-py2.py3-none-any.whl.metadata (4.2 kB) Collecting opt-einsum==3.3.0 (from paddlepaddle) Downloading opt_einsum-3.3.0-py3-none-any.whl.metadata (6.5 kB) Requirement already satisfied: protobuf>=3.20.2 in /usr/local/lib/python3.8/dist-packages (from paddlepaddle) (4.25.5) Requirement already satisfied: sympy in /usr/local/lib/python3.8/dist-packages (from x2paddle) (1.11.1) Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from x2paddle) (20.9) Collecting anyio (from httpx->paddlepaddle) Downloading anyio-4.5.2-py3-none-any.whl.metadata (4.7 kB) Requirement already satisfied: certifi in /usr/local/lib/python3.8/dist-packages (from httpx->paddlepaddle) (2024.12.14) Collecting httpcore==1.* (from httpx->paddlepaddle) Downloading httpcore-1.0.7-py3-none-any.whl.metadata (21 kB) Requirement already satisfied: idna in /usr/local/lib/python3.8/dist-packages (from httpx->paddlepaddle) (3.10) Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx->paddlepaddle) Downloading h11-0.14.0-py3-none-any.whl.metadata (8.2 kB) Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->x2paddle) (3.1.4) Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.8/dist-packages (from sympy->x2paddle) (1.2.1) Collecting sniffio>=1.1 (from anyio->httpx->paddlepaddle) Downloading sniffio-1.3.1-py3-none-any.whl.metadata (3.9 kB) Requirement already satisfied: exceptiongroup>=1.0.2 in /usr/local/lib/python3.8/dist-packages (from anyio->httpx->paddlepaddle) (1.2.2) Requirement already satisfied: typing-extensions>=4.1 in /usr/local/lib/python3.8/dist-packages (from anyio->httpx->paddlepaddle) (4.4.0) Downloading paddlepaddle-2.6.2-cp38-cp38-manylinux2014_aarch64.whl (74.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 74.3/74.3 MB 19.4 MB/s eta 0:00:00 Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB) Downloading x2paddle-1.6.0-py3-none-any.whl (325 kB) Downloading astor-0.8.1-py2.py3-none-any.whl (27 kB) Downloading decorator-5.1.1-py3-none-any.whl (9.1 kB) Downloading httpx-0.28.1-py3-none-any.whl (73 kB) Downloading httpcore-1.0.7-py3-none-any.whl (78 kB) Downloading anyio-4.5.2-py3-none-any.whl (89 kB) Downloading h11-0.14.0-py3-none-any.whl (58 kB) Downloading sniffio-1.3.1-py3-none-any.whl (10 kB) Installing collected packages: sniffio, opt-einsum, h11, decorator, astor, x2paddle, httpcore, anyio, httpx, paddlepaddle Attempting uninstall: opt-einsum Found existing installation: opt_einsum 3.4.0 Uninstalling opt_einsum-3.4.0: Successfully uninstalled opt_einsum-3.4.0 Successfully installed anyio-4.5.2 astor-0.8.1 decorator-5.1.1 h11-0.14.0 httpcore-1.0.7 httpx-0.28.1 opt-einsum-3.3.0 paddlepaddle-2.6.2 sniffio-1.3.1 x2paddle-1.6.0 requirements: AutoUpdate success ✅ 14.6s, installed 2 packages: ['paddlepaddle', 'x2paddle'] requirements: ⚠️ Restart runtime or rerun command for updates to take effect PaddlePaddle: starting export with X2Paddle 1.6.0... [ERROR] PyTorch is not installed, use "pip install torch torchvision". PaddlePaddle: export success ✅ 17.8s, saved as 'yolo11n_paddle_model/' (0.0 MB) Export complete (18.0s) Results saved to /ultralytics Predict: yolo predict task=detect model=yolo11n_paddle_model imgsz=640 Validate: yolo val task=detect model=yolo11n_paddle_model imgsz=640 data=/usr/src/ultralytics/ultralytics/cfg/datasets/coco.yaml Visualize: https://netron.app Loading yolo11n_paddle_model for PaddlePaddle inference... ERROR ❌️ Benchmark failure for PaddlePaddle: PyTorch: starting from 'yolo11n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (5.4 MB) ONNX: starting export with onnx 1.17.0 opset 16... ====== Diagnostic Run torch.onnx.export version 2.0.0a0+ec3941ad.nv23.02 ======= verbose: False, log level: Level.ERROR ======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ======================== ONNX: slimming with onnxslim 0.1.48... ONNX: export success ✅ 2.9s, saved as 'yolo11n.onnx' (10.2 MB) requirements: Ultralytics requirement ['MNN>=2.9.6'] not found, attempting AutoUpdate... WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.3.1 -> 25.0 [notice] To update, run: python3 -m pip install --upgrade pip Collecting MNN>=2.9.6 Downloading MNN-3.0.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (1.5 kB) Requirement already satisfied: numpy in /usr/local/lib/python3.8/dist-packages (from MNN>=2.9.6) (1.23.5) Downloading MNN-3.0.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (14.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.6/14.6 MB 17.3 MB/s eta 0:00:00 Installing collected packages: MNN Successfully installed MNN-3.0.3 requirements: AutoUpdate success ✅ 4.0s, installed 1 package: ['MNN>=2.9.6'] requirements: ⚠️ Restart runtime or rerun command for updates to take effect MNN: export failure ❌ 6.9s: /usr/local/lib/python3.8/dist-packages/MNN.libs/libgomp-d22c30c5.so.1.0.0: cannot allocate memory in static TLS block ERROR ❌️ Benchmark failure for MNN: /usr/local/lib/python3.8/dist-packages/MNN.libs/libgomp-d22c30c5.so.1.0.0: cannot allocate memory in static TLS block PyTorch: starting from 'yolo11n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (5.4 MB) TorchScript: starting export with torch 2.0.0a0+ec3941ad.nv23.02... TorchScript: export success ✅ 2.6s, saved as 'yolo11n.torchscript' (10.5 MB) requirements: Ultralytics requirement ['ncnn'] not found, attempting AutoUpdate... WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.3.1 -> 25.0 [notice] To update, run: python3 -m pip install --upgrade pip Collecting ncnn Downloading ncnn-1.0.20241226-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (27 kB) Requirement already satisfied: numpy in /usr/local/lib/python3.8/dist-packages (from ncnn) (1.23.5) Requirement already satisfied: tqdm in /usr/local/lib/python3.8/dist-packages (from ncnn) (4.67.1) Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from ncnn) (2.32.3) Collecting portalocker (from ncnn) Downloading portalocker-3.0.0-py3-none-any.whl.metadata (8.5 kB) Collecting opencv-python (from ncnn) Downloading opencv_python-4.11.0.86-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (20 kB) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.8/dist-packages (from requests->ncnn) (3.4.0) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->ncnn) (3.10) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->ncnn) (2.2.3) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->ncnn) (2024.12.14) Downloading ncnn-1.0.20241226-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (4.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.1/4.1 MB 12.4 MB/s eta 0:00:00 Downloading opencv_python-4.11.0.86-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (42.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.2/42.2 MB 20.5 MB/s eta 0:00:00 Downloading portalocker-3.0.0-py3-none-any.whl (19 kB) Installing collected packages: portalocker, opencv-python, ncnn Successfully installed ncnn-1.0.20241226 opencv-python-4.11.0.86 portalocker-3.0.0 requirements: AutoUpdate success ✅ 7.1s, installed 1 package: ['ncnn'] requirements: ⚠️ Restart runtime or rerun command for updates to take effect NCNN: export failure ❌ 7.1s: /usr/local/lib/python3.8/dist-packages/ncnn/../ncnn.libs/libgomp-d22c30c5.so.1.0.0: cannot allocate memory in static TLS block ERROR ❌️ Benchmark failure for NCNN: /usr/local/lib/python3.8/dist-packages/ncnn/../ncnn.libs/libgomp-d22c30c5.so.1.0.0: cannot allocate memory in static TLS block ERROR ❌️ Benchmark failure for IMX: IMX only supported for YOLOv8 Setup complete ✅ (6 CPUs, 7.2 GB RAM, 134.5/233.7 GB disk) Traceback (most recent call last): File "/usr/local/bin/yolo", line 8, in sys.exit(entrypoint()) File "/ultralytics/ultralytics/cfg/__init__.py", line 972, in entrypoint getattr(model, mode)(**overrides) # default args from model File "/ultralytics/ultralytics/engine/model.py", line 682, in benchmark return benchmark( File "/ultralytics/ultralytics/utils/benchmarks.py", line 164, in benchmark df = pd.DataFrame(y, columns=["Format", "Status❔", "Size (MB)", key, "Inference time (ms/im)", "FPS"]) UnboundLocalError: local variable 'key' referenced before assignment

Y-T-G commented 5 days ago

@iokarkan The error you posted is a result of a different earlier error. You need to post the whole thing.

iokarkan commented 5 days ago

@Y-T-G I have included the entire output of the benchmark command my original post.

Y-T-G commented 5 days ago

Reinstall torch and torchvision

iokarkan commented 5 days ago

I am not sure why this would be required when using a official docker image. Even in the docs it says to skip to the TensorRT section after running with docker.

Could you provide some insight as to why this is needed?

Y-T-G commented 5 days ago

@iokarkan I think you should open a new issue if the error is with the docker image in Jetson.

iokarkan commented 5 days ago

Yes, indeed. I think I'm coming to the conclusion that the 'key' error occurs when no benchmarks have successfully completed. This is a bug and requires its own issue.

Thanks for your help!

glenn-jocher commented 7 hours ago

Thanks for identifying this! You're correct that the 'key' error occurs when no benchmarks complete successfully. This is a known issue we're investigating. For now, please verify your Torch/Torchvision versions match our Jetson compatibility guidelines. If issues persist, feel free to open a new GitHub issue with your full error logs.

ultralytics / ultralytics

benchmark error #18965

Search before asking

Ultralytics YOLO Component

Bug

Environment

Minimal Reproducible Example

Additional

Are you willing to submit a PR?

Bug Report

Real-Time Support

Environments

Status and Updates