search

mpv-player / mpv

🎥 Command line video player
https://mpv.io
Other
28.57k stars 2.92k forks source link

Massive framedrops in mpv 0.36.0 compiled with rpi-ffmpeg 6.0 with 4k 60 Hz HEVC HDR 10bit videofiles #13055

Open mixmid opened 11 months ago

mixmid commented 11 months ago

Important Information

Provide following Information:

Sample files

Get the test video file first: $ yt-dlp -f dash-google_mediacdn-video-f4765daf https://vimeo.com/712764913 $ du -hs Samsung-and-RedBull-See-the-Unexpected-HDR-UHD-4K-(www.demolandia.net).ts\ [712764913].mp4 350M Samsung-and-RedBull-See-the-Unexpected-HDR-UHD-4K-(www.demolandia.net).ts [712764913].mp4

Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5.1@High
HDR format                               : SMPTE ST 2086, HDR10 compatible
Codec ID                                 : hvc1
Codec ID/Info                            : High Efficiency Video Coding
Duration                                 : 3 min 17 s
Bit rate                                 : 14.6 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 59.940 (60000/1001) FPS

Reproduction steps

I use specific ffmpeg libs-> /usr/lib/rpi-ffmpeg-6/lib, libplacebo lives in /usr/local/lib/aarch64-linux-gnu.

Playing the HEVC videofile 4k 60 Hz so

#!/bin/bash

LD_LIBRARY_PATH=/usr/lib/rpi-ffmpeg-6/lib:/usr/local/lib/aarch64-linux-gnu mpv --no-config --vo=gpu --gpu-context=drm --hwdec=drm \
                                        --drm-draw-plane=overlay --drm-drmprime-video-plane=primary --drm-draw-surface-size=1920x1080 \
                                        Samsung-and-RedBull-See-the-Unexpected-HDR-UHD-4K-\(www.demolandia.net\).ts\ \[712764913\].mp4

Output: 

AO: [pulse] 48000Hz stereo 2ch float
VO: [gpu] 3840x2160 drm_prime[rpi4_10]
AV: 00:00:02 / 00:03:17 (1%) A-V:  0.032 Dropped: 128

There are framedrops even massive framedrops in several seconds of playing.

And HEVC 1080p in action. 0 framedrops:

$ LD_LIBRARY_PATH=/usr/lib/rpi-ffmpeg-6/lib:/usr/local/lib/aarch64-linux-gnu mpv --no-config --vo=gpu --gpu-context=drm --hwdec=drm                                         --drm-draw-plane=overlay --drm-drmprime-video-plane=primary --drm-draw-surface-size=1920x1080                                         Ahsoka.S01E01.1080p.HEVC.x265-MeGusta\[eztv.re\].mkv
AO: [pulse] 48000Hz 5.1(side) 6ch float
VO: [gpu] 1920x1080 drm_prime[rpi4_10]
AV: 00:00:14 / 00:54:43 (0%) A-V:  0.000

I ask because these are played smoothly with no dramedrops but without sound as well, also vlc plays also without framedrops and A/V desynchronization.

And LibreElec supports for playing HEVC 4k 60 Hz somehow, i tried almost the same config but no luck. 

$ LD_LIBRARY_PATH=/usr/lib/rpi-ffmpeg-6/lib /usr/lib/rpi-ffmpeg-6/bin/ffmpeg  -no_cvt_hw -hwaccel drm -vcodec hevc -i Samsung-and-RedBull-See-the-Unexpected-HDR-UHD-4K-\(www.demolandia.net\).ts -f vout_drm /dev/null
$ LD_LIBRARY_PATH=/usr/lib/rpi-ffmpeg-6/lib ./hello_drmprime Samsung-and-RedBull-See-the-Unexpected-HDR-UHD-4K-\(www.demolandia.net\).ts

No connector ID specified.  Choosing default from list:
Connector 32 (crtc 96): type 11, 1920x1080 (chosen)
Connector 42 (crtc 0): type 11, 0x0
[hevc @ 0x5577979a40] Hwaccel V4L2 HEVC stateless V4; devices: /dev/media1,/dev/video19; buffers: src DMABuf, dst DMABuf; swfmt=rpi4_10

Expected behavior

Video plays smoothly

Actual behavior

Massive framedrops

Why not mpv 0.37?

Because these lines are in mpv log

[hwupload] Unable to find a compatible upload format for rpi4_10
[autoconvert] Failed to create HW uploader for format rpi4_10

mpv 0.37 and mpv-git still have some issues when working with rpi-ffmpeg

mixmid commented 11 months ago

Say, please, what's wrong with the template?

jeeb commented 11 months ago

Well, for one thing you did not post a full log via --log-file=mpv.log or so (both working and non-working with current master or so). So not much can be inferred from what is internally ongoing, even if one wanted to help in usage of RPI's fork of FFmpeg.

Previously the fork has had issues such as not having proper pixel format descriptors etc.

I would expect that you could maybe get 2160p60 playback when utilizing the most hard-coded rendering of such an SoC, in which case you grab the decoded result and push it onto the onboard video rendering ASIC. This would of course remove all benefits of mpv's shader based rendering. Best would be to benchmark just the decoding portion and see how quickly it handles decoding 2160p.

For the record, the pixel format the Broadcom SoC utilizes is a custom one, IIRC involving bit-banging. So most likely importing it as-is into vo=gpu or gpu-next currently would not work, as there is no reader for it.

philipl commented 11 months ago

I'm surprised it works at all. We treat it as P016 with separate planes although we do preserve the format modifiers. But it may mean that the reads are done inefficiently on the GL side.

Also, due to lack of logs, we don't know if there's too much shader work being done for 4k60.

philipl commented 11 months ago

And I completely missed the main issue. The best way you'll get a direct output path to maximise performance is by using vo=dmabuf-wayland with wayland (of course) and a wlroots based compositor (eg: sway).

llyyr commented 11 months ago

a wlroots based compositor (eg: sway).

And if you're feeling adventurous, building sway with this PR wlroots with this PR and running it with WLR_DRM_FORCE_LIBLIFTOFF=1

mixmid commented 11 months ago

Working mpv 0.36.0 log

mpv036.log

mpv 0.37.0 

[hwupload] Unable to find a compatible upload format for rpi4_10
[autoconvert] Failed to create HW uploader for format rpi4_10

mpv037.log

mixmid commented 11 months ago

I would expect that you could maybe get 2160p60 playback when utilizing the most hard-coded rendering of such an SoC, in which case you grab the decoded result and push it onto the onboard video rendering ASIC. This would of course remove all benefits of mpv's shader based rendering. Best would be to benchmark just the decoding portion and see how quickly it handles decoding 2160p.

But somehow it works:

  1. ffmpeg -no_cvt_hw -hwaccel drm -vcodec hevc -i Samsung.ts -f vout_drm /dev/null
  2. ./hello_drmprime Samsung.ts
  3. VLC works
  4. LibreElec with Kodi works
jeeb commented 11 months ago

I would expect that you could maybe get 2160p60 playback when utilizing the most hard-coded rendering of such an SoC, in which case you grab the decoded result and push it onto the onboard video rendering ASIC. This would of course remove all benefits of mpv's shader based rendering. Best would be to benchmark just the decoding portion and see how quickly it handles decoding 2160p.

But somehow it works:

1. ffmpeg  -no_cvt_hw -hwaccel drm -vcodec hevc -i Samsung.ts -f vout_drm /dev/null

2. ./hello_drmprime Samsung.ts

3. VLC works

4. LibreElec with Kodi works

These would utilize exactly the sort of interfaces that I was talking about. Just throw the decoded surface at a hard-wired interface that utilizes the built-in video rendering capabilities of the SoC.

philipl commented 11 months ago

@EmperorPenguin18 I guess you actually tried this out and know what a working configuration looks like?

The refusal to accept "rpi4_10" here makes sense in 0.37 because the hwuploader validates against the supported formats reported by the hwdec, and the hwdec cannot report "rpi4_8" or "rpi4_10" because neither of those formats exist in upstream ffmpeg. I wonder what we can reasonably do about that? Maybe there's some terrifying introspection approach where we look up the format by name to get the pixfmt value at runtime.

philipl commented 11 months ago

See #13067 which should fix the regression in 0.37

mixmid commented 11 months ago

I find out that 1080p Ahsoka.S01E01.1080p.HEVC.x265-MeGusta[eztv.re].mkv is not "HDR 10bit"

Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L4@Main
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 54 min 43 s
Bit rate                                 : 573 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 24.000 FPS

But Samsung_1080p.mp4 is HDR compatible

Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L4.1@Main
HDR format                               : SMPTE ST 2086, HDR10 compatible
Codec ID                                 : hvc1
Codec ID/Info                            : High Efficiency Video Coding
Duration                                 : 3 min 17 s
Bit rate                                 : 4 726 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 59.940 (60000/1001) FPS
mixmid commented 11 months ago

Even Samsung_360p.mp4 (640x360 px) downloaded from

$ yt-dlp -f dash-google_mediacdn-video-37897c3a https://vimeo.com/712764913

Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L3@Main
HDR format                               : SMPTE ST 2086, HDR10 compatible
Codec ID                                 : hvc1
Codec ID/Info                            : High Efficiency Video Coding
Duration                                 : 3 min 17 s
Bit rate                                 : 539HDR format  kb/s
Width                                    : 640 pixels
Height                                   : 360 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 29.970 (30000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.078

causes framedrops, it has "HDR format".

mixmid commented 11 months ago

On the other hand, there are samples like https://www.larmoire.info/jellyfish/ -- using mpv 0.37.0 with --profile=fast, framedrops happen without it.

$ LD_LIBRARY_PATH=/usr/lib/rpi-ffmpeg-6/lib:/usr/local/lib/aarch64-linux-gnu mpv --profile=fast --vo=gpu --gpu-context=drm --hwdec=drm --drm-draw-plane=overlay --drm-drmprime-video-plane=primary --drm-draw-surface-size=1920x1080 jellyfish-110-mbps-hd-hevc.mkv


$ mediainfo jellyfish-110-mbps-hd-hevc.mkv
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main@L5.1@High
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 30 s 97 ms
Bit rate                                 : 107 Mb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 29.970 (29970/1000) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Bits/(Pixel*Frame)                       : 1.729

The last means that the USB flash drive, i run it, is not slow, the Bit rate above is high enough.

philipl commented 11 months ago

As we made clear earlier, you cannot expect to get usable framerates with vo=gpu. You need to use vo=dmabuf-wayland with an appropriate wayland compositor (or vo=drm with no windowing environment)

mixmid commented 11 months ago

a wlroots based compositor (eg: sway).

And if you're feeling adventurous, building sway with this PR wlroots with this PR and running it with WLR_DRM_FORCE_LIBLIFTOFF=1

Is it needed exactly or can i use sway from Debian 12/RaspberryPi OS repo?

philipl commented 11 months ago

Is it needed exactly or can i use sway from Debian 12/RaspberryPi OS repo?

Should be fine.

mixmid commented 11 months ago

Sway just brings problems. x264 with --hwdec=v4l2m2m plays nice

screen

But with only --vo=dmabuf-wayland it goes so ->

mx@raspberrypi:~/dump/videofiles/2k $ LD_LIBRARY_PATH=/usr/lib/rpi-ffmpeg-6/lib:/usr/local/lib/aarch64-linux-gnu mpv   --vo=dmabuf-wayland Samsung_1080p.mp4
 (+) Video --vid=1 (*) (hevc 1920x1080 59.940fps)
  (+) Audio --aid=1 (*) (aac 2ch 48000Hz)
  [hwupload] no support for this hw format
  [hwupload] hardware format not supported
  [autoconvert] HW-uploading to drm_prime
  [autoconvert] Converting yuv420p10 -> nv12
  [ao/pipewire] Could not wait for initialization: Connection timed out
  AO: [pipewire] 48000Hz stereo 2ch floatp
  [hwupload] upload nv12 -> drm_prime[nv12]
  [hwupload] failed to upload frame
  Cannot convert decoder/filter output to any format supported by the output.
  Could not initialize video chain.
  Video: no video
  (Paused) A: 00:00:02 / 00:03:17 (1%)

dmabuf-only-sway.txt

With --gpu-context=wayland --vo=dmabuf-wayland --hwdec=drm ->

mx@raspberrypi:~/dump/videofiles/2k $ LD_LIBRARY_PATH=/usr/lib/rpi-ffmpeg-6/lib:/usr/local/lib/aarch64-linux-gnu mpv  --gpu-context=wayland --vo=dmabuf-wayland --hwdec=drm  Samsung_1080p.mp4
 (+) Video --vid=1 (*) (hevc 1920x1080 59.940fps)
  (+) Audio --aid=1 (*) (aac 2ch 48000Hz)
  Using hardware decoding (drm).
  [ao/pipewire] Could not wait for initialization: Connection timed out
  AO: [pipewire] 48000Hz stereo 2ch floatp
  VO: [dmabuf-wayland] 1920x1080 drm_prime[rpi4_10]
  [vo/dmabuf-wayland] Format 'P030' with modifier '(0700000000066004)' is not supported by the compositor.
  Could not initialize video chain.
  Video: no video
  (Paused) A: 00:00:01 / 00:03:17 (1%)

hwdec-sway.txt

In both cases there is only a black screen. any thoughts ?

jeeb commented 11 months ago

P030 is the bitbanged DRM format with what I hope are the modifiers for SAND 128 tiling, which are documented in https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19639 and https://gitlab.freedesktop.org/mesa/drm/-/blob/02a41cf302a69f0cd94aae96ec01d98b9398076e/include/drm/drm_fourcc.h#L1023-1067 .

So the latter is a case of the compositor saying that it cannot understand that mix, which is at least pretty clear.

mixmid commented 11 months ago

P030 is the bitbanged DRM format with what I hope are the modifiers for SAND 128 tiling, which are documented in https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19639 and https://gitlab.freedesktop.org/mesa/drm/-/blob/02a41cf302a69f0cd94aae96ec01d98b9398076e/include/drm/drm_fourcc.h#L1023-1067 .

So the latter is a case of the compositor saying that it cannot understand that mix, which is at least pretty clear.

The next iteration. These commits should already be landed in Mesa. I am using almost the latest version:


libegl-mesa0/unstable,now 23.3.1-3 arm64 [installed,automatic]
libegl1-mesa-dev/unstable,now 23.3.1-3 arm64 [installed]
libgl1-mesa-dri/unstable,now 23.3.1-3 arm64 [installed,automatic]
libglapi-mesa/unstable,now 23.3.1-3 arm64 [installed]
libglu1-mesa-dev/testing,unstable,now 9.0.2-1.1 arm64 [installed]
libglu1-mesa/testing,unstable,now 9.0.2-1.1 arm64 [installed]
libglx-mesa0/unstable,now 23.3.1-3 arm64 [installed]
mesa-drm-shim/unstable,now 23.3.1-3 arm64 [installed]
mesa-utils-bin/testing,unstable,now 8.5.0-1 arm64 [installed,automatic]
mesa-utils/testing,unstable,now 8.5.0-1 arm64 [installed]
mesa-vulkan-drivers/unstable,now 23.3.1-3 arm64 [installed,automatic]

Also i have the output from

$ weston-simple-dmabuf-feedback

weston.txt

Is it something useful?

As i see there is no support even in Weston. format P030 (0x30333050), modifier BROADCOM_SAND128 (0x700000000000004)

mixmid commented 11 months ago

It can be offtopic, but in another hand on my other system, mpv tried to do Converting vaapi[p010] -> vaapi[rgb30] on famous Samsung_HEVC_4k.mpv , because there is no support for p010.

mx@fedora:~/video/youtube$ mpv --vo=dmabuf-wayland  --hwdec=vaapi   Samsung-and-RedBull-See-the-Unexpected-HDR-UHD-4K-\(www.demolandia.net\).ts\ \[712764913\].mp4 
 (+) Video --vid=1 (*) (hevc 3840x2160 59.940fps)
 (+) Audio --aid=1 (*) (aac 2ch 48000Hz)
Cannot load libcuda.so.1
Using hardware decoding (vaapi).
[autoconvert] Converting vaapi[p010] -> vaapi[rgb30]
AO: [pipewire] 48000Hz stereo 2ch floatp
VO: [dmabuf-wayland] 3840x2160 vaapi[rgb30]
[autoconvert] Converting vaapi[p010] -> vaapi[rgb30]

This introduces another question -- how to find (for sure) supported display formats, which Wayland and a compositor knows about, as an example for vo/dmabuf-wayland/vaapi

[   0.055][v][vo/dmabuf-wayland/vaapi] Supported Wayland display format nv12: 'NV12(0100000000000002)'
[   0.055][v][vo/dmabuf-wayland/vaapi] Supported Wayland display format bgra: 'AR24(0000000000000000)'
[   0.055][v][vo/dmabuf-wayland/vaapi] Supported Wayland display format argb: 'AR24(0000000000000000)'
mixmid commented 11 months ago

I am ready to debug.

ready_to_debug

philipl commented 11 months ago

There is nothing to debug. Support in wayland compositors for all the formats you will care about is poor, and will likely remain so for a long time NV12 is supported in Sway and Weston, but I'm not sure if P010 is supported anywhere and I suspect the weird rpi P030 will wait a long time to be supported. You're trying to do something that is probably not going to be practical today.

I'd recommend using vo=drm without the desktop environment if you really need to play these files on an rpi. That is, after all, what the other applications you named are doing.

mixmid commented 11 months ago

For the record, with --vo=drm or --hwdec=drm or --gpu-context=drm. As i see, hardware acceleration isn't plugged in -- no drm_prime in VO: [drm] 3840x2160 yuv420p10, but the A/V desync's instead. Tested both on 4k and 1080p.

mx@raspberrypi:~ $ LD_LIBRARY_PATH=/usr/lib/rpi-ffmpeg-6/lib:/usr/local/lib/aarch64-linux-gnu mpv --no-config --vo=drm --gpu-context=drm --hwdec=drm --log-file=mpv_vo_drm.log '/home/mx/dump/videofiles/youtube/Samsung.ts'
 (+) Video --vid=1 (*) (hevc 3840x2160 59.940fps)
 (+) Audio --aid=1 (*) (aac 2ch 48000Hz)
[vo/drm/drm] VT_GETMODE failed: Inappropriate ioctl for device
[vo/drm/drm] Failed to set up VT switcher. Terminal switching will be unavailable.
AO: [pipewire] 48000Hz stereo 2ch floatp
VO: [drm] 3840x2160 yuv420p10
AV: 00:00:00 / 00:03:17 (0%) A-V:  0.176 Cache: 86s/147MB

Audio/Video desynchronisation detected! Possible reasons include too slow
hardware, temporary CPU spikes, broken drivers, and broken files. Audio
position will not match to the video (see A-V status field).

AV: 00:00:00 / 00:03:17 (0%) A-V:  1.879 Dropped: 9 Cache: 87s/150MB
Audio device underrun detected.
(Paused) AV: 00:00:00 / 00:03:17 (0%) A-V:  2.833 Dropped: 9 Cache: 86s/150MB

mpv_vo_drm.log

llyyr commented 11 months ago

You need profile=fast in every situation on such a weak device. mpv defaults have changed

mixmid commented 11 months ago 
mx@raspberrypi:~ $ LD_LIBRARY_PATH=/usr/lib/rpi-ffmpeg-6/lib:/usr/local/lib/aarch64-linux-gnu mpv --profile=fast --vo=drm --hwdec=auto-safe  \
                                        '/home/mx/dump/videofiles/youtube/Samsung.ts'
 (+) Video --vid=1 (*) (hevc 3840x2160 59.940fps)
 (+) Audio --aid=1 (*) (aac 2ch 48000Hz)
[vo/drm/drm] VT_GETMODE failed: Inappropriate ioctl for device
[vo/drm/drm] Failed to set up VT switcher. Terminal switching will be unavailable.
Using hardware decoding (drm-copy).
AO: [pipewire] 48000Hz stereo 2ch floatp
VO: [drm] 3840x2160 yuv420p10

Audio/Video desynchronisation detected! Possible reasons include too slow
hardware, temporary CPU spikes, broken drivers, and broken files. Audio
position will not match to the video (see A-V status field).
Consider trying `--profile=fast` and/or `--hwdec=auto-safe` as they may help.

(Paused) AV: 00:00:00 / 00:03:17 (0%) A-V:  3.617 Dropped: 11 Cache: 85s/150MB
mixmid commented 11 months ago

I'd recommend using vo=drm without the desktop environment if you really need to play these files on an rpi. That is, after all, what the other applications you named are doing.

Actively supported hwdecs: drm: requires --vo=gpu (Linux only)

drm (Direct Rendering Manager)

    Video output driver using Kernel Mode Setting / Direct Rendering Manager. Should be used when one doesn't want to install full-blown graphical environment (e.g. no X). Does not support hardware acceleration (if you need this, check the drm backend for gpu VO).

and no hardware acceleration.

I am not going to close this ticket, but will dig a little.

philipl commented 11 months ago

Yeah, that was my bad - you need vo=gpu and gpu-context=drm to have acceleration.

mixmid commented 10 months ago

Milestone N1. Tested LibreElec, mpv-drmprime was built, but the same behaviour "Unable to find a compatible upload format for rpi4_10",now that it shows the internal VideoPlayer from Kodi is needed to have more attention with DRMPRIME output, it's not a mpv problem for now.

LibreElec:/media/8gb $ mpv  --vo=gpu --gpu-context=drm --hwdec=drm Samsung_1080p.ts
 (+) Video --vid=1 (*) (hevc 1920x1080 59.940fps)
 (+) Audio --aid=1 (*) (aac 2ch 48000Hz)
[vo/gpu/drm] VT_GETMODE failed: Inappropriate ioctl for device
[vo/gpu/drm] Failed to set up VT switcher. Terminal switching will be unavailable.
Using hardware decoding (drm).
[hwupload] Unable to find a compatible upload format for rpi4_10
[autoconvert] Failed to create HW uploader for format rpi4_10
[autoconvert] HW-downloading from drm_prime
AO: [pulse] 48000Hz stereo 2ch float
VO: [gpu] 1920x1080 yuv420p10
AV: 00:00:00 / 00:03:17 (0%) A-V:  0.000
AV: 00:00:00 / 00:03:17 (0%) A-V:  0.062
AV: 00:00:00 / 00:03:17 (0%) A-V:  0.050 Dropped: 1
AV: 00:00:00 / 00:03:17 (0%) A-V:  0.033 Dropped: 2
AV: 00:00:00 / 00:03:17 (0%) A-V:  0.017 Dropped: 2
AV: 00:00:00 / 00:03:17 (0%) A-V:  0.004 Dropped: 3
AV: 00:00:00 / 00:03:17 (0%) A-V:  0.003 Dropped: 4
AV: 00:00:00 / 00:03:17 (0%) A-V:  0.002 Dropped: 4
Exiting... (Quit)
AV: 00:00:00 / 00:03:17 (0%) A-V:  0.002 Dropped: 4
philipl commented 10 months ago

Are you using mpv master with the patch I committed?

mixmid commented 10 months ago

Nope, LibreELEC is using the official release of 0.37.

https://github.com/LibreELEC/LibreELEC.tv/blob/master/packages/addons/addon-depends/multimedia-tools-depends/mpv-drmprime/package.mk

I was able to get 4k 30Hz in Kodi on RaspberryPi OS 12, but 60Hz are not taken.

philipl commented 10 months ago

Then none of this is a surprise. You need that patch in mpv for hwdec to work.

mixmid commented 10 months ago

MPV in some way can be used as an external player, but Kodi has it's own internal VideoPlayer which works as others. 

1. ffmpeg  -no_cvt_hw -hwaccel drm -vcodec hevc -i Samsung.ts -f vout_drm /dev/null

2. ./hello_drmprime Samsung.ts

3. VLC works

4. LibreElec with Kodi works
philipl commented 10 months ago

What are you trying to learn at this stage? Your investigation identified the original regression, which is now fixed in master. Continuing to test 0.37 will just tell us what we already know.

mixmid commented 10 months ago

What are you trying to learn at this stage? Your investigation identified the original regression, which is now fixed in master. Continuing to test 0.37 will just tell us what we already know.

HEVC SDRs play nice, see movie/Ahsoka.S01E01.1080p.HEVC.x265-MeGusta[eztv.re].mkv, it's 1080p.

But HDR10 files play with significant framedrops, even so small like Samsung_360p.mp4 (640x360 px). drm_prime[rpi4_10] here.


 mx@raspberrypi:~/dump/videofiles/2k $ LD_LIBRARY_PATH=/usr/lib/rpi-ffmpeg-rpi6/lib:/usr/local/lib/aarch64-linux-gnu mpv --log-file=small-360p --profile=fast --vo=gpu --gpu-context=drm --hwdec=drm /home/mx/dump/videofiles/2k/Samsung_360px.mp4 
 (+) Video --vid=1 (*) (hevc 640x360 29.970fps)
 (+) Audio --aid=1 (*) (aac 2ch 48000Hz)
[vo/gpu/drm] VT_GETMODE failed: Inappropriate ioctl for device
[vo/gpu/drm] Failed to set up VT switcher. Terminal switching will be unavailable.
Using hardware decoding (drm).
AO: [pipewire] 48000Hz stereo 2ch floatp
VO: [gpu] 640x360 drm_prime[rpi4_10]
(Paused) AV: 00:00:05 / 00:03:17 (3%) A-V:  0.000 Dropped: 36 Cache: 191s/22MB

small-360p.log

emveepee commented 10 months ago

@mixmid how are you compiling ffmpeg to use with mpv and get drm or drm-copy working? I can only get those with the distro 5.1.4 ffmpeg and self compiled mpv 0.37 master or the distro mpv but my self compiles of ffmpeg with mpv all don't work. This is with --vo-gpu running RPi OS lite (no wayland). I have tried with deb-src ffmpeg, ffmpeg master and rpi-ffmpeg master. When it doesn't work with my ffmpeg compiles I don't see and I don't know where they come from.

[   0.550][d][vo/gpu/opengl]   sand128 => 2 planes 2x2 8/0 [r8/rg8] (r/gb) [unorm]
[   0.550][d][vo/gpu/opengl]   sand64_10 => 2 planes 2x2 16/-6 [r16/rg16] (r/gb) [unorm]
[   0.550][d][vo/gpu/opengl]   sand64_16 => 2 planes 2x2 16/0 [r16/rg16] (r/gb) [unorm]
[   0.550][d][vo/gpu/opengl]   rpi4_8
[   0.550][d][vo/gpu/opengl]   rpi4_10
mixmid commented 10 months ago

I use this file as an example like PKG_URL, download http://ffmpeg.org/releases/ffmpeg-${PKG_VERSION}.tar.xz" https://github.com/LibreELEC/LibreELEC.tv/tree/master/packages/multimedia/ffmpeg

After i apply this patch on common ffmpeg from ffmpeg.org: https://github.com/LibreELEC/LibreELEC.tv/blob/master/packages/multimedia/ffmpeg/patches/rpi/ffmpeg-001-rpi.patch

This patch (ffmpeg-001-rpi.patch) is almost the same as https://github.com/jc-kynesim/rpi-ffmpeg/tree/dev/6.0/rpi_import_1

Next thing. Configuring so, see -> --prefix=/usr/lib/rpi-ffmpeg-rpi6. And make -j4, make install:

CFLAGS="-march=native -mtune=native" CXXFLAGS="-march=native -mtune=native" ./configure --prefix=/usr/lib/rpi-ffmpeg-rpi6 --disable-network --disable-debug --disable-outdev=fbdev --disable-outdev=oss --disable-doc --disable-bsfs --disable-ffprobe --disable-sdl2 --disable-stripping --disable-thumb --disable-mmal --enable-sand --enable-v4l2-request --enable-libdrm --enable-v4l2_m2m --enable-epoxy --enable-libudev --enable-vout-drm --enable-libx264 --enable-libx265 --enable-gpl --enable-shared --enable-libmp3lame --enable-libopus --enable-libdav1d --enable-libvorbis --enable-libvpx --enable-libxvid --enable-opengl --disable-vdpau --disable-nvdec --disable-nvenc --enable-outdev=vout_drm --enable-hwaccels --enable-encoder=wrapped_avframe --enable-muxer=null --enable-encoder=rawvideo --enable-muxer=rawvideo --enable-muxer=image2 --enable-muxer=md5 --enable-muxer=framemd5 --enable-neon

Also this command is needed then you build mpv to choose rpi ffmpeg libs:

PKG_CONFIG_PATH=/usr/lib/rpi-ffmpeg-rpi6/lib/pkgconfig meson setup build

Next i use LD_PRELOAD allowing ffmpeg from repos to live together, latest libplacebo lives in /usr/local/lib/aarch64-linux-gnu

$ LD_LIBRARY_PATH=/usr/lib/rpi-ffmpeg-rpi6/lib:/usr/local/lib/aarch64-linux-gnu mpv --profile=fast --vo=gpu --gpu-context=drm --hwdec=drm FILE.mp4

emveepee commented 10 months ago

Thanks a lot, drm and drm-copy both work now. I guess it was the LE patch and extra options it allows.

I took your configure pruned it down for using with mpv-build

./use-ffmpeg-custom n6.0.1 cd ffmpeg git apply ~/ffmpeg-001-rpi.patch

These are my ffmpeg_options, not sure if they are all required.

--enable-libdrm
--enable-sand
--enable-v4l2-request
--enable-opengl
--enable-outdev=vout_drm
--enable-hwaccels
--enable-v4l2_m2m

I wonder why deb-src didn't work but I like being on v6 anyway?

mixmid commented 10 months ago 
--enable-libdrm
--enable-sand
--enable-v4l2-request
--enable-opengl
--enable-outdev=vout_drm
--enable-hwaccels
--enable-v4l2_m2m

They almost are required for hardware acceleration, for h.264 at least. Good to keep them on. As an example, official ffmpeg does not contain code and configure option --enable-sand.

I wonder why deb-src didn't work but I like being on v6 anyway?

But ffmpeg from repos does not contain rpi patches.

emveepee commented 10 months ago

Thank,s right now I am working on the RPi5 so can't test h264 or h262 but they are not dropping too many frames anyway, that subset is working well. I did need libdav1d and I am sure I will need others

Since deb-src is generic Debian, I was hoping to find out why ffmpeg + mpv on Raspberry PI OS works with drm by looking at their configure options I thought maybe there was a trick but I guess not.

I have a secondary problem with a segfault on on hevc files using drm and drm-copy but things are way better than yesterday so thanks again. It is somehow linked to these journal entries.

[21702.307225] rpivid 1000800000.codec: rpivid_h265_start: (1920x1088)
[21702.311864] rpivid 1000800000.codec: SPS changed
[21702.311874] rpivid 1000800000.codec: PPS changed
[21704.207282] rpivid 1000800000.codec: rpivid_h265_stop
mixmid commented 10 months ago

@emveepee,

Please, can you post results of playing Samsung_4k.ts and Samsung_360px.mp4 from https://drive.google.com/drive/folders/1molnO-0kgR1BMmT82BNTMM6mqNvlSHOD?usp=sharing

using

mpv --profile=fast --vo=gpu --gpu-context=drm --hwdec=drm FILE.mp4

i guess that hdr 10bit stuff causes framedrops and cuts off hardware acceleration.

mixmid commented 10 months ago

I have a secondary problem with a segfault on on hevc files using drm and drm-copy but things are way better than yesterday so thanks again. It is somehow linked to these journal entries.

Then you need to compile mpv with

PKG_CONFIG_PATH=/usr/lib/rpi-ffmpeg-rpi6/lib/pkgconfig meson setup build --buildtype=debug

And use gdb $ LD_LIBRARY_PATH=/usr/lib/rpi-ffmpeg-6/lib:/usr/local/lib/aarch64-linux-gnu gdb --args mpv --profile=fast --vo=gpu --gpu-context=drm --hwdec=drm Samsung_360px.mp4

and (gdb) r

After the segfault is caught eventually use

(gdb) bt

And post the backtrace here.