[Detector Support]: TensorRT CUDA Version Problem

[JoshuaPK]

Discussed in https://github.com/blakeblackshear/frigate/discussions/11424

^{Originally posted by **JoshuaPK** May 18, 2024} ### Describe the problem you are having I am trying to set up Frigate using a TensorRT detector with CUDA. I have configured and verified the CUDA driver, libraries, and container tools with my 3050. When I start Frigate, it gets to the point where it starts to generate yolov7-320.trt, and then it fails with an error indicating a problem with the CUDA driver vs. library version (i.e. error 35). ### Version Unsure- haven't got that far yet ### Frigate config file ```yaml mqtt: enabled: False cameras: dummy_camera: # <--- this will be changed to your actual camera later enabled: False ffmpeg: inputs: - path: rtsp://127.0.0.1:554/rtsp roles: - detect ``` ### docker-compose file or Docker CLI command ```yaml version: "3.9" services: frigate: container_name: frigate #privileged: true # this may not be necessary for all setups restart: unless-stopped image: ghcr.io/blakeblackshear/frigate:stable-tensorrt shm_size: "4gb" # update for your cameras based on calculation above deploy: resources: reservations: devices: - driver: nvidia device_ids: ['0'] count: 1 capabilities: [gpu] volumes: - /etc/localtime:/etc/localtime:ro - /srv/vcs/frigate/config:/config - /srv/vcs/frigate/media:/media/frigate - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear target: /tmp/cache tmpfs: size: 1000000000 ports: - "5000:5000" - "8554:8554" # RTSP feeds - "8555:8555/tcp" # WebRTC over tcp - "8555:8555/udp" # WebRTC over udp environment: FRIGATE_RTSP_PASSWORD: "password" ``` ### Relevant log output ```shell Frigate: [frigate] | Creating yolov7-320.cfg and yolov7-320.weights [frigate] | [frigate] | Done. [frigate] | 2024-05-18 16:01:34.222576297 [INFO] Starting go2rtc healthcheck service... [frigate] | [frigate] | Generating yolov7-320.trt. This may take a few minutes. [frigate] | Traceback (most recent call last): File "/usr/local/src/tensorrt_demos/yolo/onnx_to_tensorrt.py", line 214, in main() File "/usr/local/src/tensorrt_demos/yolo/onnx_to_tensorrt.py", line 202, in main engine = build_engine( File "/usr/local/src/tensorrt_demos/yolo/onnx_to_tensorrt.py", line 112, in build_engine with trt.Builder(TRT_LOGGER) as builder, builder.create_network(*EXPLICIT_BATCH) as network, trt.OnnxParser(network, TRT_LOGGER) as parser: TypeError: pybind11::init(): factory function returned nullptr [frigate] | [05/18/2024-16:01:37] [TRT] [W] Unable to determine GPU memory usage [frigate] | [05/18/2024-16:01:37] [TRT] [W] Unable to determine GPU memory usage [frigate] | [05/18/2024-16:01:37] [TRT] [W] CUDA initialization failure with error: 35. Please check your CUDA installation: http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html [frigate] | Loading the ONNX file... [frigate] | Available tensorrt models: ls: cannot access '*.trt': No such file or directory nvidia-smi: Sat May 18 16:31:13 2024 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce RTX 3050 Off | 00000000:01:00.0 Off | N/A | | 34% 39C P0 N/A / 70W | 0MiB / 6144MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | No running processes found | +-----------------------------------------------------------------------------------------+ ``` ### Operating system Other Linux ### Install method Docker Compose ### Coral version CPU (no coral) ### Any other information that may be helpful I am using Podman instead of Docker on Almalinux 9.4.

[NickM-27]

issues are for feature requests