search

jellyfin / jellyfin-media-player

Jellyfin Desktop Client
GNU General Public License v2.0
3.28k stars 323 forks source link

Hardware decoding not working in Flatpak for 4K HDR files #231

Open iainsaxonhome opened 2 years ago

iainsaxonhome commented 2 years ago

Describe the bug Playback of 4K HDR videos is choppy running around 10-15 FPS and intel_gpu_top will show activity on the VideoEnhance/0 engine whereas the latest version of MPV via Flatpak with correct decoding support runs smoothly and uses no VideoEnhance/0 resources.

Playing back a 4K high-bitrate file using only SDR works correctly and plays back smoothly so it appears to be specifically HDR that causes the problem. 4K is simply because that's when HDR was introduced.

I've attached the log for the working version of MPV and Jellyfin Media Player where it lags.

The detail that I think makes the difference is the version of OpenGL that each detects and it appears that JMP lacks Vulkan support and is referring to an older version of OpenGL that also lacks the ES features.

MPV (smooth):

[cplayer] Configuration: waf configure --prefix=/app --enable-libmpv-shared --disable-build-date --disable-manpage-build --disable-alsa --enable-sdl2 --enable-libarchive --enable-dvbin --enable-cdda --enable-dvdnav --enable-shaderc --enable-vulkan
[cplayer] List of enabled features: asm cdda cplayer cplugins cuda-hwaccel cuda-interop debug-build drm dvbin dvdnav egl egl-drm egl-helpers egl-x11 ffmpeg ffmpeg-aviocontext-bytes-read ffnvcodec gbm gbm.h gl gl-wayland glibc-thread-name glob glob-posix gpl iconv jack javascript jpeg lcms2 libarchive libass libavdevice libbluray libdl libm libmpv-shared libplacebo librt linux-fstatfs linux-input-event-codes lua luajit memfd_create optimize plain-gl posix posix-or-mingw pthreads pulse rubberband sdl2 sdl2-audio sdl2-gamepad sdl2-video shaderc shaderc-shared sixel stdatomic uchardet vaapi vaapi-drm vaapi-egl vaapi-vulkan vaapi-wayland vaapi-x-egl vaapi-x11 vapoursynth vdpau vector vt.h vulkan wayland wayland-protocols x11 xv zimg zlib
[cplayer] Reading config file /app/etc/mpv/encoding-profiles.conf
...
[vo/gpu/opengl] EGL_VERSION=1.5
[vo/gpu/opengl] EGL_VENDOR=Mesa Project
[vo/gpu/opengl] EGL_CLIENT_APIS=OpenGL OpenGL_ES 
[vo/gpu/opengl] Trying to create Desktop OpenGL context.
[vo/gpu/opengl] Choosing visual EGL config 0x15, visual ID 0x21
[vo/gpu/opengl] GL_VERSION='4.6 (Core Profile) Mesa 21.3.8 (git-813ee839be)'
[vo/gpu/opengl] Detected desktop OpenGL 4.6.
[vo/gpu/opengl] GL_VENDOR='Intel'
[vo/gpu/opengl] GL_RENDERER='Mesa Intel(R) HD Graphics 500 (APL 2)'
[vo/gpu/opengl] GL_SHADING_LANGUAGE_VERSION='4.60'
...
[vd] Codec list:
[vd]     hevc - HEVC (High Efficiency Video Coding)
[vd]     hevc_v4l2m2m (hevc) - V4L2 mem2mem HEVC decoder wrapper
[vd]     hevc_cuvid (hevc) - Nvidia CUVID HEVC decoder
[vd] Opening decoder hevc
[vd] Looking at hwdec hevc-nvdec...
[vo/gpu] Loading hwdec driver 'vaapi-egl'
[vo/gpu/vaapi-egl] using VAAPI EGL interop
[vo/gpu/vaapi-egl] Trying to open a x11 VA display...
[vo/gpu/vaapi-egl/vaapi] Initialized VAAPI: version 1.12
[vo/gpu/vaapi-egl] Going to probe surface formats (may log bogus errors)...
[vo/gpu/vaapi-egl] Done probing surface formats.
[vo/gpu] Loading hwdec driver 'cuda-nvdec'
[vo/gpu/cuda-nvdec] Failed to load CUDA symbols
[vo/gpu] Loading failed.
[vo/gpu] Loading hwdec driver 'drmprime-drm'
[vo/gpu/drmprime-drm] Failed to retrieve DRM fd from native display.
[vo/gpu] Loading failed.
[vd] Could not create device.
[vd] Looking at hwdec hevc-vaapi...
[vd] Trying hardware decoding via hevc-vaapi.
[vd] Selected codec: hevc (HEVC (High Efficiency Video Coding))

JMP (choppy):

2022-03-31 06:57:18 [ DEBUG ] PlayerComponent.cpp @ 585 - cplayer: Configuration: waf configure --prefix=/app --enable-libmpv-shared --disable-cplayer --disable-build-date --disable-manpage-build --enable-vaapi --enable-vdpau --enable-cuda-hwaccel --enable-pulse --enable-alsa --disable-tv --enable-uchardet 
2022-03-31 06:57:18
[ DEBUG ] PlayerComponent.cpp @ 585 - cplayer: List of enabled features: alsa asm cplugins cuda-hwaccel cuda-interop debug-build drm egl egl-drm egl-helpers egl-x11 ffmpeg ffnvcodec gbm gbm.h gl gl-wayland glibc-thread-name glob glob-posix gpl iconv jpeg lcms2 libarchive libass libavdevice libdl libm libmpv-shared librt linux-fstatfs linux-input-event-codes memfd_create optimize plain-gl posix posix-or-mingw pthreads pulse stdatomic uchardet vaapi vaapi-drm vaapi-egl vaapi-wayland vaapi-x-egl vaapi-x11 vdpau vt.h wayland wayland-protocols x11 xv zlib 
...
2022-03-31 06:57:18 [ DEBUG ] PlayerComponent.cpp @ 585 - libmpv_render: GL_VERSION='3.0 Mesa 21.1.8 (git-127871cd1e)' 
2022-03-31 06:57:18 [ DEBUG ] PlayerComponent.cpp @ 585 - libmpv_render: Detected desktop OpenGL 3.0. 
2022-03-31 06:57:18 [ DEBUG ] PlayerComponent.cpp @ 585 - libmpv_render: GL_VENDOR='Intel Open Source Technology Center' 
2022-03-31 06:57:18 [ DEBUG ] PlayerComponent.cpp @ 585 - libmpv_render: GL_RENDERER='Mesa DRI Intel(R) HD Graphics 500 (APL 2)' 
2022-03-31 06:57:18 [ DEBUG ] PlayerComponent.cpp @ 585 - libmpv_render: GL_SHADING_LANGUAGE_VERSION='1.30' 
...
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd: Codec list: 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd:     hevc - HEVC (High Efficiency Video Coding) 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd:     hevc - HEVC (High Efficiency Video Coding) 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd:     hevc_v4l2m2m (hevc) - V4L2 mem2mem HEVC decoder wrapper 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd:     hevc_cuvid (hevc) - Nvidia CUVID HEVC decoder 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd: Opening decoder hevc 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd: Looking at hwdec hevc-nvdec... 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd: Could not create device. 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd: Looking at hwdec hevc-vaapi... 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd: Could not create device. 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd: Looking at hwdec hevc-vdpau... 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd: Could not create device. 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd: Looking at hwdec hevc-nvdec-copy... 
2022-03-31 06:57:23 [ ERROR ] PlayerComponent.cpp @ 591 - ffmpeg: AVHWDeviceContext: Cannot load libcuda.so.1 
2022-03-31 06:57:23 [ ERROR ] PlayerComponent.cpp @ 591 - ffmpeg: AVHWDeviceContext: Could not dynamically load CUDA 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd: Could not create device. 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd: Looking at hwdec hevc-vaapi-copy... 
2022-03-31 06:57:23 [ INFO  ] JS: nowplaying event: init 
2022-03-31 06:57:23 [ INFO  ] JS: Requesting url without automatic networking: http://192.168.1.2:8096/Sessions/Playing 
2022-03-31 06:57:23 [ INFO  ] JS: nowplaying event: playbackstart 
2022-03-31 06:57:23 [ INFO  ] JS: nowplaying event: playbackstart 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vaapi: Initialized VAAPI: version 1.8 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd: Trying hardware decoding via hevc-vaapi-copy. 
2022-03-31 06:57:23 [ DEBUG ] PlayerComponent.cpp @ 585 - vd: Selected codec: hevc (HEVC (High Efficiency Video Coding))

This is related to https://github.com/jellyfin/jellyfin-media-player/issues/182.

It also seems related to https://github.com/jellyfin/jellyfin-media-player/issues/179 however I am not sure what the successful fix was for that issue.

Is it possible to update the JMP Flatpak manifest to use similar flags and dependencies to a more recent version of MPV? MPV: https://github.com/flathub/io.mpv.Mpv/blob/master/io.mpv.Mpv.yml#L431 JMP: https://github.com/flathub/com.github.iwalton3.jellyfin-media-player/blob/master/com.github.iwalton3.jellyfin-media-player.json#L53

I hope this is enough information and I've followed the right debugging path :) .

To Reproduce Steps to reproduce the behavior:

  1. Set Jellyfin Media Player client settings hardwareDecoding to decode.enable support (not decode.copy)
  2. Play back a 4K video file with HDR
  3. Jellyfin's MPV will use vaapi-copy and not vaapi requiring more CPU power than normal
  4. The video will be slow and choppy

Expected behavior The embedded version of MPV should use the non-copy version of the VAAPI decoder.

Desktop:

Additional context I suspect the detail that is missing is Vulkan support that is present in the latest version of MPV via Flatpak.

Logs jellyfinmediaplayer.log mpv.log

iainsaxonhome commented 2 years ago

Also found this is related to https://github.com/jellyfin/jellyfin-media-player/issues/139.

I forgot to mention in my previous comment that JMP is using Mesa (org.freedesktop.Platform.GL.default) branch 20.08 version 21.1.8 whereas MPV is using branch 21.08 version 21.3.8.

Like issue #139 I think the key issue is the embedded MPV can't use the vaapi decoder and is falling back to vaapi-copy.

(Sorry, didn't mean to close ticket, accidentally tapped on mobile while typing).

Krupski19 commented 2 years ago

Has there been any progress into resolving this issue? I am on the latest Jellyfin media player deb package on Pop OS, and I am experiencing the same issue with 4K SDR content as well.

I am running on Gen 9 Intel integrated graphics, and with hardware decoding in JMP set to decode.enabled, I get dropped frames and frame rate drops when trying to play 4K content. Looking at intel_gpu_top, I am getting low encoder utilization and very high CPU utilization. Checking the JMP logs shows in fact that JMP is using vaapi-copy as the decoding device rather than vaapi, as it could not create the device, resulting in very poor playback.

If I copy the stream URL, and open it up in MPV, the video plays perfectly! There are no dropped frames or frame rate issues at all, looking at intel_gpu_top shows high encoder utilization, and there is practically no CPU utilization at all!

Is there something we could add to our JMP config files or MPV config files to force JMP to use vaapi?

Looking at the logs also shows something else that may be interesting, libmpv-render fails to load hedec-driver 'vaapi-egl', stating 2022-07-02 12:12:48 [ DEBUG ] PlayerComponent.cpp @ 588 - libmpv_render/vaapi-egl: VAAPI hwdec only works with OpenGL or Vulkan backends. Could this have anything to do with why vaapi is not working correctly under JMP? jellyfinmediaplayer.log

jluttine commented 2 years ago

I have the same issue: JMP is unable to use vaapi and falls back to vaapi-copy, but vaapi works fine with plain MPV if I copy the stream URL there.

Could this MPV issue be related: https://github.com/mpv-player/mpv/issues/10424

I must say, I didn't really understand how to use the solution they had in that issue, but maybe it could be applied here too somehow?

jluttine commented 2 years ago

Hmm.. Running JMP with:

QT_XCB_GL_INTEGRATION="xcb_egl" jellyfinmediaplayer

Seems to enable vaapi:

2022-10-27 14:45:16 [ DEBUG ] PlayerComponent.cpp @ 588 - libmpv_render: Loading hwdec driver 'vaapi'
2022-10-27 14:45:16 [ DEBUG ] PlayerComponent.cpp @ 588 - libmpv_render/vaapi: using EGL dmabuf interop
2022-10-27 14:45:16 [ DEBUG ] PlayerComponent.cpp @ 588 - libmpv_render/vaapi: Trying to open a x11 VA display...
2022-10-27 14:45:16 [ DEBUG ] PlayerComponent.cpp @ 588 - libmpv_render/vaapi/vaapi: Initialized VAAPI: version 1.15
2022-10-27 14:45:16 [ DEBUG ] PlayerComponent.cpp @ 588 - libmpv_render/vaapi: Going to probe surface formats (may log bogus errors)...
2022-10-27 14:45:16 [ DEBUG ] PlayerComponent.cpp @ 588 - libmpv_render/vaapi: Done probing surface formats.

EDIT: Yep, works perfectly for me now. I wonder if this should/could be added to JMP itself somewhere in the code?

Krupski19 commented 2 years ago

I can confirm that running jellyfinmediaplayer using QT_XCB_GL_INTEGRATION="xcb_egl" jellyfinmediaplayer does indeed enable vaapi decode for me and playback of 4k video content for me using Jellyfin Media Player is far improved and on par now with MPV.

tony1016 commented 9 months ago

QT_XCB_GL_INTEGRATION="xcb_egl" jellyfinmediaplayer

I can confirm too

SandyRodgers-2017 commented 5 months ago

How do is set QT_XCB_GL_INTEGRATION="xcb_egl" on jellyfinmediaplayer flatpak?