TensorRT not working in Linux - cannot find libcudnn.so.8

[fl-la]

Describe the bug

PopOS 20.04 with Cuda 11-6 and tensorrt 8. Installed obs as well as the plugin through flatpak. When switchting to gpu it throws this message:

error: /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1106 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_tensorrt.so with error: libcudnn.so.8: cannot open shared object file: No such file or directory

Expected behavior

Switchting to gpu

Screenshots

If applicable, add screenshots to help explain your problem.

Log and Crash Report

Upload your OBS log file here (Help>Log Files>Show Log Files) to help us find cause of the problem.

If OBS is crashing, upload the crash report here as well (Help>Crash Reports>Show Crash Reports).

14:27:30.754: Using EGL/X11 14:27:30.754: CPU Name: Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz 14:27:30.754: CPU Speed: 2400.001MHz 14:27:30.755: Physical Cores: 8, Logical Cores: 16 14:27:30.755: Physical Memory: 64059MB Total, 16655MB Free 14:27:30.755: Kernel Version: Linux 6.0.12-76060012-generic 14:27:30.755: Flatpak Branch: stable 14:27:30.755: Flatpak Arch: x86_64 14:27:30.755: Flatpak Runtime: runtime/org.kde.Platform/x86_64/6.4 14:27:30.755: App Extensions: 14:27:30.755: - com.obsproject.Studio.Plugin.BackgroundRemoval 14:27:30.755: - com.obsproject.Studio.Locale 14:27:30.755: Runtime Extensions: 14:27:30.755: - org.freedesktop.Platform.GL.Debug.default 14:27:30.755: - org.freedesktop.Platform.GL.default 14:27:30.755: - org.freedesktop.Platform.GL.nvidia-470-161-03 14:27:30.755: - org.freedesktop.Platform.VAAPI.Intel 14:27:30.755: - org.freedesktop.Platform.openh264 14:27:30.755: - org.gtk.Gtk3theme.Breeze 14:27:30.755: - org.gtk.Gtk3theme.Pop-dark 14:27:30.755: - org.kde.PlatformTheme.QGnomePlatform 14:27:30.755: - org.kde.PlatformTheme.QGnomePlatform.Sources 14:27:30.755: - org.kde.WaylandDecoration.QGnomePlatform-decoration 14:27:30.755: - org.kde.WaylandDecoration.QGnomePlatform_decoration.Sources 14:27:30.755: - org.kde.Platform.Locale 14:27:30.755: - org.freedesktop.Platform.GL.Debug.default 14:27:30.755: - org.freedesktop.Platform.GL.default 14:27:30.755: Flatpak Framework Version: 1.8.5 14:27:30.755: Desktop Environment: pop:GNOME (pop) 14:27:30.755: Session Type: x11 14:27:30.755: Window System: X11.0, Vendor: The X.Org Foundation, Version: 1.20.13 14:27:30.757: Qt Version: 6.4.3 (runtime), 6.4.3 (compiled) 14:27:30.757: Portable mode: false 14:27:30.821: OBS 29.1.3 (linux) 14:27:30.821: --------------------------------- 14:27:30.821: --------------------------------- 14:27:30.821: audio settings reset: 14:27:30.821: samples per sec: 48000 14:27:30.821: speakers: 2 14:27:30.821: max buffering: 960 milliseconds 14:27:30.821: buffering type: dynamically increasing 14:27:30.827: --------------------------------- 14:27:30.827: Initializing OpenGL... 14:27:32.122: Loading up OpenGL on adapter NVIDIA Corporation Quadro T2000 with Max-Q Design/PCIe/SSE2 14:27:32.122: OpenGL loaded successfully, version 3.3.0 NVIDIA 470.161.03, shading language 3.30 NVIDIA via Cg compiler 14:27:32.166: --------------------------------- 14:27:32.166: video settings reset: 14:27:32.166: base resolution: 1920x1080 14:27:32.166: output resolution: 1280x720 14:27:32.166: downscale filter: Bicubic 14:27:32.166: fps: 30/1 14:27:32.166: format: NV12 14:27:32.166: YUV mode: Rec. 709/Partial 14:27:32.166: NV12 texture support not available 14:27:32.166: P010 texture support not available 14:27:32.166: Audio monitoring device: 14:27:32.166: name: Standard 14:27:32.166: id: default 14:27:32.166: --------------------------------- 14:27:32.190: No AJA devices found, skipping loading AJA UI plugin 14:27:32.190: Failed to initialize module 'aja-output-ui.so' 14:27:32.210: No AJA devices found, skipping loading AJA plugin 14:27:32.210: Failed to initialize module 'aja.so' 14:27:32.217: Failed to load 'en-US' text for module: 'decklink-captions.so' 14:27:32.225: Failed to load 'en-US' text for module: 'decklink-output-ui.so' 14:27:32.231: A DeckLink iterator could not be created. The DeckLink drivers may not be installed 14:27:32.231: Failed to initialize module 'decklink.so' 14:27:32.506: [pipewire] No captures available 14:27:32.546: [obs-browser]: Version 2.21.1 14:27:32.546: [obs-browser]: CEF Version 103.0.5060.134 (runtime), 103.0.0-5060-shared-textures_143.2591+g4204d54+chromium-103.0.5060.134 (compiled) 14:27:32.559: NVENC supported 14:27:32.562: VAAPI: API version 1.18 14:27:32.562: FFmpeg VAAPI H264 encoding supported 14:27:32.564: FFmpeg VAAPI HEVC encoding supported 14:27:32.629: [obs-websocket] [obs_module_load] you can haz websockets (Version: 5.2.3 | RPC Version: 1) 14:27:32.629: [obs-websocket] [obs_module_load] Qt version (compile-time): 6.4.3 | Qt version (run-time): 6.4.3 14:27:32.629: [obs-websocket] [obs_module_load] Linked ASIO Version: 101201 14:27:32.638: [obs-websocket] [obs_module_load] Module loaded. 14:27:32.690: Failed to load 'de-DE' text for module: 'obs-backgroundremoval.so' 14:27:32.690: [obs-backgroundremoval] plugin loaded successfully (version 1.0.1) 14:27:32.690: --------------------------------- 14:27:32.690: Loaded Modules: 14:27:32.690: obs-backgroundremoval.so 14:27:32.690: text-freetype2.so 14:27:32.690: rtmp-services.so 14:27:32.690: obs-x264.so 14:27:32.690: obs-websocket.so 14:27:32.690: obs-vst.so 14:27:32.690: obs-transitions.so 14:27:32.690: obs-outputs.so 14:27:32.690: obs-libfdk.so 14:27:32.690: obs-filters.so 14:27:32.690: obs-ffmpeg.so 14:27:32.690: obs-browser.so 14:27:32.690: linux-v4l2.so 14:27:32.690: linux-pulseaudio.so 14:27:32.690: linux-pipewire.so 14:27:32.690: linux-jack.so 14:27:32.690: linux-capture.so 14:27:32.690: image-source.so 14:27:32.690: frontend-tools.so 14:27:32.690: decklink-output-ui.so 14:27:32.690: decklink-captions.so 14:27:32.690: --------------------------------- 14:27:32.691: ==== Startup complete =============================================== 14:27:32.742: All scene data cleared 14:27:32.742: ------------------------------------------------ 14:27:32.744: pulse-input: Server name: 'pulseaudio 13.99.1' 14:27:32.745: pulse-input: Audio format: s16le, 44100 Hz, 2 channels 14:27:32.745: pulse-input: Started recording from 'bluez_sink.20_64_DE_E1_D3_99.a2dp_sink.monitor' (default) 14:27:32.745: [Loaded global audio device]: 'Desktop-Audio' 14:27:32.746: pulse-input: Server name: 'pulseaudio 13.99.1' 14:27:32.746: pulse-input: Audio format: s24le, 48000 Hz, 2 channels 14:27:32.746: pulse-input: Sample format s24le not supported by OBS,using float32le instead for recording 14:27:32.746: pulse-input: Started recording from 'alsa_input.usb-Generic_USB_Audio_200901010001-00.HiFi__hw_Dock__source' (default) 14:27:32.746: [Loaded global audio device]: 'Mikrofon/AUX-Audio' 14:27:32.794: v4l2-input: Start capture from /dev/video4 14:27:32.794: v4l2-input: Input: 0 14:27:32.864: v4l2-input: Resolution: 1920x1080 14:27:32.864: v4l2-input: Pixelformat: MJPG 14:27:32.864: v4l2-input: Linesize: 0 Bytes 14:27:32.864: v4l2-input: Framerate: 30.00 fps 14:27:32.866: v4l2-input: /dev/video4: select timeout set to 166666 (5x frame periods) 14:27:33.011: Model models/mediapipe.onnx input 0: name input_1:0 shape (4 dim) 1 x 144 x 256 x 3 14:27:33.011: Model models/mediapipe.onnx output 0: name segment:0 shape (4 dim) 1 x 144 x 256 x 2 14:27:33.012: [obs-backgroundremoval] Allocated 110592 sized float-array for input 0 14:27:33.012: [obs-backgroundremoval] Allocated 73728 sized float-array for output 0 14:27:33.016: Switched to scene 'meeting' 14:27:33.016: ------------------------------------------------ 14:27:33.016: Loaded scenes: 14:27:33.016: - scene 'Kamera_hintergrund_manuell': 14:27:33.016: - source: 'Bildschirmkamera_raw' (v4l2_input) 14:27:33.016: - source: 'Ausblenden_Fenster_abschnitt' (color_source_v3) 14:27:33.016: - source: 'Ausblenden_Fenster_abschnitt' (color_source_v3) 14:27:33.016: - source: 'Ausblenden_Fenster_abschnitt' (color_source_v3) 14:27:33.016: - source: 'Ausblenden_Fenster_abschnitt' (color_source_v3) 14:27:33.016: - source: 'Ausblenden_Fenster_abschnitt' (color_source_v3) 14:27:33.016: - source: 'Ausblenden_Fenster_abschnitt' (color_source_v3) 14:27:33.016: - source: 'Ausblenden_Fenster_abschnitt' (color_source_v3) 14:27:33.016: - scene 'Meeting_camera_template': 14:27:33.016: - source: 'Kamera_meeting' (group) 14:27:33.016: - filter: 'Greenscreen_optimiert' (chroma_key_filter_v2) 14:27:33.016: - source: 'Kamera_hintergrund_manuell' (scene) 14:27:33.016: - scene 'Kamera_background_removal': 14:27:33.016: - source: 'Background_remover' (group) 14:27:33.016: - filter: 'Background Removal' (background_removal) 14:27:33.016: - filter: 'Chroma Key' (chroma_key_filter_v2) 14:27:33.016: - source: 'Bildschirmkamera_raw' (v4l2_input) 14:27:33.016: - scene 'Meeting_HintergrundUniBW': 14:27:33.016: - source: 'WeisserHintergrund 2' (color_source_v3) 14:27:33.016: - source: 'Hintergrund_UniBW 2' (group) 14:27:33.016: - source: 'FI_Code' (image_source) 14:27:33.016: - source: 'UniBW_Logo' (image_source) 14:27:33.016: - source: 'Meeting_camera_template' (scene) 14:27:33.016: - source: 'Kamera_background_removal' (scene) 14:27:33.016: - scene 'Meeting_Hintergrund_sonstige': 14:27:33.084: - source: 'Background_clean 2 2' (image_source) 14:27:33.084: - source: 'Background_library 2 2' (image_source) 14:27:33.084: - source: 'Meeting_camera_template' (scene) 14:27:33.084: - source: 'Kamera_background_removal' (scene) 14:27:33.084: ------------------------------------------------ 14:27:34.256: adding 42 milliseconds of audio buffering, total audio buffering is now 42 milliseconds (source: Desktop-Audio) 14:27:34.256: 14:28:00.692: User switched to scene 'Kamera_background_removal' 14:28:14.376: /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1106 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_tensorrt.so with error: libcudnn.so.8: Kann die Shared-Object-Datei nicht öffnen: Datei oder Verzeichnis nicht gefunden 14:28:14.376: 14:32:30.488: Model models/mediapipe.onnx input 0: name input_1:0 shape (4 dim) 1 x 144 x 256 x 3 14:32:30.488: Model models/mediapipe.onnx output 0: name segment:0 shape (4 dim) 1 x 144 x 256 x 2 14:32:30.489: [obs-backgroundremoval] Allocated 110592 sized float-array for input 0 14:32:30.489: [obs-backgroundremoval] Allocated 73728 sized float-array for output 0 14:32:37.134: /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1106 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_tensorrt.so with error: libcudnn.so.8: Kann die Shared-Object-Datei nicht öffnen: Datei oder Verzeichnis nicht gefunden

Desktop (please complete the following information):

• OS: [e.g. iOS] PopOS 20.04
• Browser: [e.g. chrome, safari]
• Plugin Version: [e.g. 0.5.1] 1.0.1
• OBS Version: [e.g. 28.02] 29.1.3

Additional context

Add any other context about the problem here.

[umireon]

Can you install cuDNN and try again?

[Grief]

@umireon which version of tensorrt has to be installed? I've installed tensorrt 8.6.1.6-1 + cuda12.0 on ubuntu following the official way of the installation (downloading deb packages from nvidia site) and obs started to crash as soon as I select tensorrt as an option in background removal plugin settings. It crashes with [1] 179554 IOT instruction obs and there is nothing suspicious in the logs

[umireon]

https://onnxruntime.ai/docs/execution-providers/TensorRT-ExecutionProvider.html#requirements TensorRT must be 8.6 and CUDA must be 11.8. CUDA 12.0 is not supported.

[Grief]

@umireon I've installed tensorrt=8.6.1.6-1+cuda11.8 but obs still crashes with IOT instruction obs as soon as I switch from cpu to tensorrt :(

[umireon]

@Grief Can you file a new issue with a filled template to track your problem?

[Erotemic]

I'm also getting this issue with flatpak on Ubuntu 22.04.

error: [obs-backgroundremoval] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1209 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_tensorrt.so with error: libcudnn.so.8: cannot open shared object file: No such file or directory

error: [obs-backgroundremoval] Failed to create ONNXRuntime session. Error code: 5
info: [obs-backgroundremoval] Background filter updated
info: [obs-backgroundremoval] Background Removal Filter Options:
info: [obs-backgroundremoval]   Source: Background Removal
info: [obs-backgroundremoval]   Model: models/mediapipe.onnx
info: [obs-backgroundremoval]   Inference Device: tensorrt
info: [obs-backgroundremoval]   Num Threads: 4
info: [obs-backgroundremoval]   Enable Threshold: true
info: [obs-backgroundremoval]   Threshold: 0.300000
info: [obs-backgroundremoval]   Contour Filter: 0.130000
info: [obs-backgroundremoval]   Smooth Contour: 0.800000
info: [obs-backgroundremoval]   Feather: 0.050000
info: [obs-backgroundremoval]   Mask Every X Frames: 1
info: [obs-backgroundremoval]   Enable Image Similarity: false
info: [obs-backgroundremoval]   Image Similarity Threshold: 35.000000
info: [obs-backgroundremoval]   Blur Background: 0
info: [obs-backgroundremoval]   Enable Focal Blur: false
info: [obs-backgroundremoval]   Blur Focus Point: 0.100000
info: [obs-backgroundremoval]   Blur Focus Depth: 0.000000
info: [obs-backgroundremoval]   Disabled: true
info: [obs-backgroundremoval]   Model file path: /app/plugins/share/obs/obs-plugins/obs-backgroundremoval/models/mediapipe.onnx
error: [obs-backgroundremoval] Model is not initialized
error: [obs-backgroundremoval] Model is not initialized
error: [obs-backgroundremoval] Model is not initialized

I have a 3090 GPU. I'm not sure how to get flatpak to recognize it or install what is needed in the flatpak environment.

[royshil]

Failed to load library libonnxruntime_providers_tensorrt.so with error: libcudnn.so.8: cannot open shared object file: No such file or directory

Looks like libcudnn doesn't exist, or at least not on the PATH

[Erotemic]

Looks like libcudnn doesn't exist, or at least not on the PATH

That makes sense, but how do I get libcudnn in the context of the OBS flatpak install? It wouldn't see a system version if I installed it, right? I assume it would need to be in the flatpak sandbox somewhere?

[umireon]

I don't think you can use GPU feature of this plugin with Flatpak

[Erotemic]

I don't think you can use GPU feature of this plugin with Flatpak

That's a bummer given that the official OBS distribution on Ubuntu is now flatpak. Is there anything that could be done to remedy this? The CPU model is great, but it seems a bit inaccurate. My hope is that one of the GPU models will be more powerful and give a cleaner segmentation. It would also be nice to offload that computation.

I've been attempting to build OBS from source on Ubuntu 22.04 so I could then try this plugin from source and get the GPU feature, but I'm running into issues with it being unable to find Python for some reason (which I'm guessing is the cmake python discovery tools not playing nice with pyenv).

[umireon]

the official OBS distribution on Ubuntu is now flatpak.

Is this true? Where the OBS project stated this?

Is there anything that could be done to remedy this?

I think Flatpak cannot support CUDA and no remedy will be available.

I've been attempting to build OBS from source on Ubuntu 22.04 so I could then try this plugin from source and get the GPU feature, but I'm running into issues with it being unable to find Python for some reason (which I'm guessing is the cmake python discovery tools not playing nice with pyenv).

Please file a new issue for building problems.

[Erotemic]

Is this true? Where the OBS project stated this?

No, I made a mistake. There is still an official PPA that provides the latest version. I'm not sure what happened, but last time I tried to use that I got an old version of obs, which is where my misconception came from.

I'll submit more issues if I can determine they are not a "my-environment" problem.