Playback stuttering with --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample on videos with framerate close to monitor refresh rate.

TheSiman commented 1 year ago

Important Information

Provide following Information:

mpv version: git master
Linux Distribution and Version: Fedora 37 beta KDE Plasma spin
Source of the mpv binary: mpv-build - git master
Window Manager and version: kwin 5.25.5
GPU driver and version: NVIDIA 515.76
Possible screenshot or video of visual glitches

https://user-images.githubusercontent.com/73035614/193739255-ceb830b3-6dbd-491d-93f9-f55588de4dde.mp4

Reproduction steps

The issue doesn't appear with --no-config. I am able to reproduce the issue with --no-config --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample on most 60fps videos (my display refresh rate is also 60Hz) regardless of resolution. 24fps videos don't have this effect. The problem also doesn't occur with --gpu-api=opengl or --vo=gpu. Some 60fps videos have this glitching throughout the entire video, some only in certain parts, but all of them have this issue with stats enabled (both temporary and permanent)

Expected behavior

Playback is not glitching while not dropping frames.

Actual behavior

Playback is glitching while not dropping frames.

Log file

https://0x0.st/oJMi.txt

Sample files

https://www.libde265.org/downloads-videos/

haasn commented 1 year ago

I can't reproduce this. Can you try some of the following settings?

--vulkan-async-compute=false
--vulkan-swap-mode=immediate --no-audio (this may disable DS though)

Apart from that, not sure what else to try. It seems kinda random / strange.

TheSiman commented 1 year ago

All settings below were used with --no-config --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample.

With --vulkan-async-compute=no the window doesn't render at all, I can hear audio and alt tab to and from the player, my mouse disappears when "hovering over" where the windows "is". Log file: https://0x0.st/oJ3H.txt

With --vulkan-swap-mode=immediate --no-audio:

https://user-images.githubusercontent.com/73035614/194115602-a7780f3d-5a56-486a-8c43-ade6293db49e.mp4

Log file: https://0x0.st/oJ3c.txt

With --vulkan-swap-mode=immediate the audio runs steadily 1x speed and video speed is somehow affected by stats (there is also no sync between video and audio):

https://user-images.githubusercontent.com/73035614/194115664-4f538bce-b4c3-4001-8eed-9c24401f0f41.mp4

Log file: https://0x0.st/oJ3S.txt

With --no-audio the video glitches only with stats (or at least visibly to me), and significantly less than without --no-audio:

https://user-images.githubusercontent.com/73035614/194116762-a11f6e1a-1277-4f4a-bc35-454a13197cee.mp4

Log file: https://0x0.st/oJ3j.txt

haasn commented 1 year ago

With --vulkan-async-compute=no the window doesn't render at all, I can hear audio and alt tab to and from the player, my mouse disappears when "hovering over" where the windows "is". Log file: https://0x0.st/oJ3H.txt

Oh dear, I can reproduce this on my end - it causes a full GPU hang. I'll investigate it.

No idea what's causing the other issues though.

haasn commented 1 year ago

Oh dear, I can reproduce this on my end - it causes a full GPU hang. I'll investigate it.

I fixed this bug in libplacebo master, and also a related potential issue concerning synchronization that I noticed while investigating it. Maybe it's relevant to your issue somehow?

Do you also get the same weird flickering with --video-sync=audio --scale=bicubic_fast?

TheSiman commented 1 year ago

Do you also get the same weird flickering with --video-sync=audio --scale=bicubic_fast?

No, with these two and the previous options (minus display-resample) it doesn't glitch. A-V also stays locked at 0.000

I fixed this bug in libplacebo master, and also a related potential issue concerning synchronization that I noticed while investigating it. Maybe it's relevant to your issue somehow?

Should I rebuild with mpv-build?

haasn commented 1 year ago

Should I rebuild with mpv-build?

Yes. (Make sure to use ./use-libplacebo-master)

haasn commented 1 year ago

No, with these two and the previous options (minus display-resample) it doesn't glitch. A-V also stays locked at 0.000

Can you make a (short!!) log reproducing the flickering issue with --msg-level=vo=trace --log-file=mpv-sucks.log?

TheSiman commented 1 year ago

Ok, did a rebuild with ./use-libplacebo-master followed by ./update (is this step necessary, or would ./rebuild do the same thing anyway?) and ./rebuild. Here's the log, it's about 14k lines, but it's as fast as I could open stats https://0x0.st/oJYp.log

haasn commented 1 year ago

@TheSiman Thanks. Can you make another log of the same but with --video-sync=audio?

TheSiman commented 1 year ago

https://0x0.st/oJUT.log

haasn commented 1 year ago

Well, the good news is that it's doing the right thing. The bad news is that this doesn't explain the bug.

garamond13 commented 1 year ago

Having exactly the same problem on windows 10, nvidia. It only happens in fullscreen and only if the shift + i info is on or osd is visible or the mouse cursor is visible (so when I move the mouse). The cause for this, at least for me, is high value in --swapchain-depth, if I set it to 1 problem completely disappears.

TheSiman commented 1 year ago

I also have no glitching with --swapchain-depth=1.

I have tried experimenting with different values:

1,2 => no glitching
3 => extreme glitching (i assume this was the default; Edit: this is indeed the default)
4 => slight glitching
5-8 => no glitching (at least not visible to me anyway)

So, at least in my case, the issue doesn't seem to be a high swapchain-depth.

@garamond13 what values have you tried, do you have the same pattern perhaps?

garamond13 commented 1 year ago

1,2 no glitching 3,4 seems most more then 4, seems less glitchy than 3 and 4

crypticC0der commented 1 year ago

any update on this?

haasn commented 1 year ago

I need to be able to reproduce the issue to effectively debug it, I fear

Obegg commented 1 year ago

I can reproduce this issue, using gpu-next with gpu-api=vulkan and 60fps youtube video. Windows 10, latest drivers and build. Is there anything I can do to help @haasn solve it?

Edit: Something I've noticed is that the "Redraw" section on "frame timing" stats page doesn't update, only the "Fresh" timing are updated every second.

Edit 2: According to what users suggested here as a temporary fix is to use swapchain-depth=1, but according to my testing, while it is a fix to the main issue here, it introduces another problem, now the vsync-jitter fluctuates between 0.017 to 0.060, which makes it seem like frame drop/frame skip every once in a while, which is not an ideal way of watching, and quite annoying and distracting, I tried using vulkan-swap-mode=fifo to counter that, but failed, so... I'll stick to gpu-api=d3d11 for now, unless someone has other ideas on how to fix it?

Obegg commented 1 year ago

I did some more testing (hopefully something from this will help solve this issue). @haasn

Test 1

mpv --no-config --fs --log-file=test1.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample --swapchain-depth=8 --glsl-shader="~~/FSRCNNX_x2_16-0-4-1.glsl" "60fps.mp4"

Heavy stuttering (even without using osc) + a lot of frame drops test1.txt

Test 2

mpv --no-config --fs --log-file=test2.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample --swapchain-depth=8 --glsl-shader="~~/FSRCNNX_x2_8-0-4-1.glsl" "60fps.mp4"

No stuttering (even with using osc), No frame drops test2.txt

Test 3

mpv --no-config --fs --log-file=test3.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample --swapchain-depth=4 --glsl-shader="~~/FSRCNNX_x2_16-0-4-1.glsl" "60fps.mp4"

Heavy stuttering (even without using osc) + a lot of frame drops test3.txt

Test 4

mpv --no-config --fs --log-file=test4.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample --swapchain-depth=1 --glsl-shader="~~/FSRCNNX_x2_16-0-4-1.glsl" "60fps.mp4"

No stuttering (even with using osc), but with frame drops test4.txt

Test 5

mpv --no-config --fs --log-file=test5.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample --swapchain-depth=2 --glsl-shader="~~/FSRCNNX_x2_16-0-4-1.glsl" "60fps.mp4"

No visible stuttering (even with using osc), but with frame drops test5.txt

Test 6

mpv --no-config --fs --log-file=test6.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample --swapchain-depth=3 --glsl-shader="~~/FSRCNNX_x2_16-0-4-1.glsl" "60fps.mp4"

Stuttering only when using osc, with frame drops test6.txt

Test 7

mpv --no-config --fs --log-file=test7.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample --swapchain-depth=4 --glsl-shader="~~/FSRCNNX_x2_16-0-4-1.glsl" "60fps.mp4"

Heavy stuttering (even without using osc), with frame drops test7.txt

Test 8

mpv --no-config --fs --log-file=test8.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample --swapchain-depth=5 --glsl-shader="~~/FSRCNNX_x2_16-0-4-1.glsl" "60fps.mp4"

Heavy stuttering (even without using osc), with frame drops test8.txt

Test 9

mpv --no-config --fs --log-file=test9.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample --swapchain-depth=6 --glsl-shader="~~/FSRCNNX_x2_16-0-4-1.glsl" "60fps.mp4"

Heavy stuttering (even without using osc), with frame drops test9.txt

Test 10

mpv --no-config --fs --log-file=test10.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample --swapchain-depth=7 --glsl-shader="~~/FSRCNNX_x2_16-0-4-1.glsl" "60fps.mp4"

Heavy stuttering (even without using osc), with frame drops test10.txt

Test 11

mpv --no-config --fs --log-file=test11.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample --swapchain-depth=8 --glsl-shader="~~/FSRCNNX_x2_16-0-4-1.glsl" "60fps.mp4"

Heavy stuttering (even without using osc), with frame drops test11.txt

Test 12

mpv --no-config --fs --log-file=test12.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample --swapchain-depth=8 --glsl-shader="~~/FSRCNNX_x2_8-0-4-1.glsl" "60fps.mp4"

No stuttering (even with using osc), no frame drops test12.txt

Test 13

mpv --no-config --fs --log-file=test13.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=audio --swapchain-depth=8 --glsl-shader="~~/FSRCNNX_x2_16-0-4-1.glsl" "60fps.mp4"

Heavy stuttering (even without using osc), with frame drops test13.txt

Test 14

mpv --no-config --fs --log-file=test14.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=audio --swapchain-depth=8 --glsl-shader="~~/FSRCNNX_x2_8-0-4-1.glsl" "60fps.mp4"

No stuttering (even with using osc), no frame drops test14.txt

Test 15

mpv --no-config --fs --log-file=test15.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample --swapchain-depth=8 "60fps.mp4"

No stuttering (even with using osc), no frame drops test15.txt

Dudemanguy commented 1 year ago

The conclusion from that is simply that your GPU isn't powerful enough to run FSRCNNX 16 at 60fps which is hardly surprising (mine can't either). It's not related to the OP.

Obegg commented 1 year ago

I am aware of the fact that my GPU isn't power enough, but I think it is related since there are some cases where there is no stuttering, even while using FSRCNNX 16, for example test6.

I did some more testing, with and without shaders, here: (I now believe it's related to shaders and scale and etc) If needed more testing let me know

Test 16

mpv --no-config --opengl-pbo --vd-lavc-skipframe=none --fs --log-file=test16.txt --gpu-debug --vo=gpu-next --vd-lavc-skiploopfilter=none --tscale=lanczos --cscale=ewa_lanczos --gpu-api=vulkan --video-sync=display-resample --swapchain-depth=8 --vulkan-swap-mode=fifo --vulkan-async-transfer --vulkan-async-compute --scaler-lut-size=10 --target-colorspace-hint=yes --scale=ewa_lanczos --cscale=ewa_lanczos --dither-depth=12 --dither=ordered "bbb-1920x1080-cfg02.mkv"

Stuttering only with osc, no frame drops test16.txt

Test 17

mpv --no-config --vd-lavc-skipframe=none --fs --log-file=test17.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample --vulkan-swap-mode=fifo --vulkan-async-transfer "bbb-1920x1080-cfg02.mkv"

No stuttering, no frame drops test17.txt

Test 18

mpv --no-config --fs --log-file=test18.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample "bbb-1920x1080-cfg02.mkv" --scale=ewa_lanczos --cscale=ewa_lanczos --dscale=ewa_lanczos --tscale=lanczos

Stuttering only with osc, no frame drops test18.txt

Test 19

mpv --no-config --fs --log-file=test19.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample "bbb-1920x1080-cfg02.mkv"

No stuttering (even with osc), no frame drops test19.txt

Test 20

mpv --no-config --scale=ewa_lanczos --fs --log-file=test20.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample "bbb-1920x1080-cfg02.mkv"

Stuttering only with osc, no frame drops test20.txt

Test 21 (same as 20, but with `--msg-level=vo=trace`)

mpv --no-config --msg-level=vo=trace --scale=ewa_lanczos --fs --log-file=test21.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample "bbb-1920x1080-cfg02.mkv"

Stuttering only with osc, no frame drops test21.txt

Test 22 (same as 21, but without `--scale=ewa_lanczos`)

mpv --no-config --msg-level=vo=trace --fs --log-file=test22.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample "bbb-1920x1080-cfg02.mkv"

No stuttering (even with osc), no frame drops test22.txt

Test 23 (same as 21, but with `--vulkan-async-compute=no`)

mpv --no-config --vulkan-async-compute=no --msg-level=vo=trace --scale=ewa_lanczos --fs --log-file=test23.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=display-resample "bbb-1920x1080-cfg02.mkv"

No stuttering (even with osc), no frame drops (only a few at the start of the video file) test23.txt

Test 24 (same as 21, but with `--video-sync=audio`)

mpv --no-config --msg-level=vo=trace --scale=ewa_lanczos --fs --log-file=test24.txt --gpu-debug --vo=gpu-next --gpu-api=vulkan --video-sync=audio "bbb-1920x1080-cfg02.mkv"

Stuttering only with osc, no frame drops test24.txt

christoph-heinrich commented 1 year ago

Tuzap on IRC ran into this problem when running mpv --no-config --vo=gpu-next --gpu-api=vulkan --fs --cscale=ewa_lanczos and enabling stats with a 1080p 60fps 4:2:0 video on both a 1080p 60Hz and a 2160p 60Hz display, with the problem being more pronounced on the 2160p one. They use Windows 10 with a Nvidia GPU.

@quietvoid and I tried to reproduce the issue, both on linux with AMD GPUs but were unable to. With OP being on linux too, it seems like this is only a problem for Nvidia GPUs.

monarc99 commented 1 year ago

tried on my linux system with nvidia:

can confirm the issue with software decoding.

Works much better with different

swapchain depth: (--swapchain-depth=8) hw decoding: (--hwdec=auto) or disabled direct renerding: (--vd-lavc-dr=no)

Obegg commented 11 months ago

@haasn Now that you have NVIDIA GPU can you try to reproduce this issue?

mpv-player / mpv