Dropping all frames (again)

[raffaem]

I have the exact same problem as #29.

I use Hyprland with NVIDIA proprietary drivers and mpvpaper is dropping all frames:

It worked fine with nouveau.

mpv is able to play the video.

I think it may be due to the fact that mpv uses vo=gpu and mpvpaper uses vo=libmpv?

How can I get full debug output from mpv?

[GhostNaN]

Did you read through https://github.com/GhostNaN/mpvpaper/issues/29 ? It's either a nvidia driver, mpv or hyperland issue. You could narrow it down by trying to see if mpvpaper works on sway. If it works on sway, I would just upstream the issue to mpv

I think it may be due to the fact that mpv uses vo=gpu and mpvpaper uses vo=libmpv?

No, see here as to why: https://github.com/GhostNaN/mpvpaper/issues/10

How can I get full debug output from mpv?

I don't think that would be much help in this case. Other than running mpvpaper in a debugger, you won't find where it's getting hold up.

[raffaem]

It's either a nvidia driver, mpv or hyperland issue.

I don't see how. mpv can play the video just fine, on hyprland with nvidia driver.

[GhostNaN]

True and there might be something going on with libmpv. But there are more moving parts going on here.

For example, it could be an issue with the hyperland if the wl_surface_frame won't callback. It could be a mpv issue if only the libmpv render is having this issue. And it also might be a driver issue because changing the driver seems to fix the issue.

But now I'm curious, I wonder if the old mpvpaper render loop works. It's only other thing I could think of that could be mpvpaper's fault. This was the last commit before the new render loop was implemented: https://github.com/GhostNaN/mpvpaper/commit/f65700a3ecc9ecd8ca501e18a807ee18845f9441 You could clone and compile the files from here: https://github.com/GhostNaN/mpvpaper/tree/f65700a3ecc9ecd8ca501e18a807ee18845f9441

[raffaem]

With that commit it says this:

❯ mpvpaper '*' "wallpaper.webm"
[-] :/ sorry about this but we can't seem to find any output.

[GhostNaN]

Yes as mpvpaper only supported 1 monitor back then, hence the render loop rewrite.

[raffaem]

So we pick one random project amon mpv, hyprland, wlroots, nvidia drivers, and send the bug report there? I don't understand

[GhostNaN]

First, what was the result with the old render loop? Second, I'm not going to lie, I'm not sure.

I'm thinking mpv will be your best bet if you can rule out it's not hyperland's fault by try a different wlroots compositor(probably sway). Otherwise, I don't have a clue unless I know where mpvpaper is getting caught up on your Nvidia drivers.

[raffaem]

Didn't I report the result with the old render loop here? https://github.com/GhostNaN/mpvpaper/issues/32#issuecomment-1317238004

[GhostNaN]

No, you got that error because you used "*" to select all monitors. That commit was before that was implemented so you need to specify a monitor manually. An example from the man page: mpvpaper DP-2 /path/to/video If you don't know what your monitor is labeled as use this: mpvpaper -v a a And check the not selected warnings.

[raffaem]

And how I was supposed to know it was not implemented yet?

It works perfectly fine with the old render loop.

[GhostNaN]

Not saying you should of known, perhaps I should of been more clear when I said https://github.com/GhostNaN/mpvpaper/issues/32#issuecomment-1317240830

Anyways, Now that we know it was the render loop. What specifically is causing this behavior for Nvidia GPU drivers is the question. I have an old Nvidia laptop I'll see if it has the same issue.

[raffaem]

Bisected to 0475df808f302c11c2d69ad1bd32fd2acb9cf2cf

[GhostNaN]

Yeah, that was the first commit toward the new render loop.

[GhostNaN]

My laptop was too old to test, only the Intel IGPU is working. If mpvpaper didn't error out during EGL initialization we can rule out those changes.

I wonder if it's just the polling that's broken. Go here on the new version and replace this: https://github.com/GhostNaN/mpvpaper/blob/5dcdfc70ce542626e052b58278840e9018b329d9/src/main.c#L1052-L1101 With this:

    // Main Loop
    while (true) {
        if (wl_display_flush(state.display) == -1 && errno != EAGAIN)
            break;

        if (wl_display_dispatch(state.display) == -1)
            break;

        mpv_render_context_update(mpv_glcontext);

        // Draw frame for all outputs
        struct display_output *output;
        wl_list_for_each(output, &state.outputs, link) {
            // Redraw immediately if not waiting for frame callback
            if (output->frame_callback == NULL) {
                if (output->egl_window && output->egl_surface)
                    render(output);
            } else {
                output->redraw_needed = true;
            }
        }
    }

Then recompile. This will remove all the polling and mpvpaper will render as fast as your pc can.

[raffaem]

No, it's still bugged.

Can you send .patch files?

[GhostNaN]

Ok, so it's not the polling. That's good. Here's the modified main.c Polling removed main.tar.gz

For my next guess, maybe it has to with implementing wl_output v4 https://github.com/GhostNaN/mpvpaper/commit/a5a47502ca6d206a9bfff8468392ef11b2499cd8 Try this version before the wl_output v4 commit: https://github.com/GhostNaN/mpvpaper/tree/18c2f4131cd1ac00aa35f82c0651af17af71fa76

Or it could be that I updated the GLAD libraries https://github.com/GhostNaN/mpvpaper/commit/6b0f7a8b9a18e757dbcba44b9d34afe59a9094e2 Try this version before the GLAD update: https://github.com/GhostNaN/mpvpaper/tree/32b9f9a1121b4a14afd646a5cdbfdf9924ab12e3

I'm just throwing darts and seeing what sticks.

[raffaem]

With the code here there is no "dropped" error anymore, but the screen is completely black

[raffaem]

Try this version before the wl_output v4 commit: https://github.com/GhostNaN/mpvpaper/tree/18c2f4131cd1ac00aa35f82c0651af17af71fa76

No. Black screen and "Dropped" error message

[raffaem]

Try this version before the GLAD update: https://github.com/GhostNaN/mpvpaper/tree/32b9f9a1121b4a14afd646a5cdbfdf9924ab12e3

No. Black screen and "Dropped" error message.

I think proper debug logging capabilities should be implemented in mpvpaper.

[raffaem]

I have obtained a debug log from mpv by issuing mpvpaper -o "--log-file=output.txt".

There are a bunch of

[   0.428][v][vo/libmpv] mpv_render_context_render() not being called or stuck.

output.txt

[GhostNaN]

I think proper debug logging capabilities should be implemented in mpvpaper.

I don't disagree, it's always been a bit lacking. It's just tedious to implement. I'm going to add a new level of verbose that will make this process easier in the future.

https://github.com/GhostNaN/mpvpaper/issues/32#issuecomment-1318152640 Great idea. Although, I'm not surprised about that. It wouldn't be dropping frames if this was being called: https://github.com/GhostNaN/mpvpaper/blob/5dcdfc70ce542626e052b58278840e9018b329d9/src/main.c#L155-L159

With the code here there is no "dropped" error anymore, but the screen is completely black

Interesting, try also with just 1 monitor. It's calling the render() function, but it seems it doesn't want to display. The only thing added to the render() function that could matter is this: https://github.com/GhostNaN/mpvpaper/blob/5dcdfc70ce542626e052b58278840e9018b329d9/src/main.c#L150-L153 Remove/comment out these^ lines and see if it displays onto just 1 monitor.

It could also be that Nvidia doesn't like having multiple EGL Surfaces.

[raffaem]

nope, doesn't work

[GhostNaN]

Here's what I've come up to improve debugging: verbose main

While it's not a full on debug mode, I added a bunch more error printing and verbose messages. And there's more I could add, but this is what I have for now.

Let's see what messages you get with -v and the new -vv flag.

[raffaem]

❯ ./build/mpvpaper -vv -o "--loop=inf" HDMI-A-1 "/home/raffaele/Videos/wallpaper.webm"
[*] Verbose Level 2 enabled
[+] Connected to Wayland compositor
[*] Output HDMI-A-1 (Samsung Electric Company C24F390 H4LT305839) selected
[*] OpenGL 4.6 EGL context created
[+] EGL initialized
 (+) Video --vid=1 (*) (vp9 3840x2160 25.000fps)
 (+) Audio --aid=1 --alang=eng (*) (opus 2ch 48000Hz)
[*] Loaded /home/raffaele/Videos/wallpaper.webm
[+] MPV initialized
[+] HDMI-A-1 setup is complete, ready to start rendering
[*] HDMI-A-1 is ready for MPV to render the next frame
AO: [pipewire] 48000Hz stereo 2ch floatp
VO: [libmpv] 3840x2160 yuv420p
[*] MPV is ready to render the next frame for HDMI-A-1
[*] HDMI-A-1 is ready for MPV to render the next frame
[*] HDMI-A-1 is ready for MPV to render the next frame
AV: 00:00:04 / 00:02:00 (3%) A-V:  0.000 Dropped: 97

[*] Exiting mpvpaper

[GhostNaN]

No errors... I thought for sure there would be an issue with the EGL surface. And it's getting held up in the main while(true) {} render loop. Which we already knew with https://github.com/GhostNaN/mpvpaper/issues/32#issuecomment-1318142239

But there is 1 new thing here I didn't think would occur. It made it the render() function 3 times with when it said [*] HDMI-A-1 is ready for MPV to render the next frame from the frame_handle_done() function. And once with [*] MPV is ready to render the next frame for HDMI-A-1 in the main while(true) {} render loop. Although, that still doesn't point the exact source of the issue.

I've looked a lot at the diff between mpvpaper version 1.2.1 and 1.3 The only thing I could think of that's broken is the multi EGL surfaces needed for multi monitor (despite no errors)

Commits:

1. https://github.com/GhostNaN/mpvpaper/commit/19f7129cdbfa9fb1e17d105da5dec243f130514a
2. https://github.com/GhostNaN/mpvpaper/commit/0475df808f302c11c2d69ad1bd32fd2acb9cf2cf
3. https://github.com/GhostNaN/mpvpaper/commit/781320f5c0b19ed37a0d274720a045a1cd60e32f

Are the main suspects. The first was when the EGL surfaces was split among the outputs The second reworked how the EGL contexts was used The third added the polling to save processing time.

If you don't want to do this next step, I don't blame you. You would have to clone and compile from each commit tree and find the 1 that breaks for you. Either way, thank you so much for your patience!

[raffaem]

Bisected to 0475df8

Didn't I bisect it?

I don't understand

[GhostNaN]

Didn't I bisect it?

I don't know what you mean by "bisect", like "fork"? Are you saying https://github.com/GhostNaN/mpvpaper/commit/0475df808f302c11c2d69ad1bd32fd2acb9cf2cf is when it started dropping frames/breaking? If so, this narrows down the issue substantially!

[raffaem]

https://git-scm.com/docs/git-bisect

[GhostNaN]

Thanks, I learned something today. I'll go narrow down the possibilities and give you a modified main.c again later.

[GhostNaN]

Here it is reordered_egl_main.tar.gz All I did was reorder the EGL initialization similarly to the way it was before the new render loop. I'm thinking this is were the Nvidia driver screws up. As a proof of concept, this version only works with 1 monitor so don't specify '*'.

[raffaem]

It displays the first frame of the video and then gets stuck on that.

❯ ./build/mpvpaper -vv -o "--loop=inf" HDMI-A-1 "/home/raffaele/Videos/wallpaper.webm"
[*] Verbose Level 2 enabled
[+] Connected to Wayland compositor
[*] Output HDMI-A-1 (Samsung Electric Company C24F390 H4LT305839) selected
[*] OpenGL 4.6 EGL context created
[+] EGL initialized
 (+) Video --vid=1 (*) (vp9 3840x2160 25.000fps)
 (+) Audio --aid=1 --alang=eng (*) (opus 2ch 48000Hz)
[*] Loaded /home/raffaele/Videos/wallpaper.webm
[+] MPV initialized
[*] MPV is ready to render the next frame for HDMI-A-1
AO: [pipewire] 48000Hz stereo 2ch floatp
VO: [libmpv] 3840x2160 yuv420p
[*] MPV is ready to render the next frame for HDMI-A-1
[*] HDMI-A-1 is ready for MPV to render the next frame
[*] HDMI-A-1 is ready for MPV to render the next frame
[*] MPV is ready to render the next frame for HDMI-A-1
[*] MPV is ready to render the next frame for HDMI-A-1
AV: 00:00:04 / 00:02:00 (4%) A-V:  0.323 Dropped: 112

Audio/Video desynchronisation detected! Possible reasons include too slow
hardware, temporary CPU spikes, broken drivers, and broken files. Audio
position will not match to the video (see A-V status field).

AV: 00:00:07 / 00:02:00 (6%) A-V:  0.000 Dropped: 178

[*] Exiting mpvpaper

[GhostNaN]

Short of removing the polling again, here's all the changes I can make with EGL reordered_egl_main_2.tar.gz

[raffaem]

Again, renders the first frame but then gets stuck on that XD

❯ ./build/mpvpaper -vv -o "--loop=inf" HDMI-A-1 "/home/raffaele/Videos/wallpaper.webm"
[*] Verbose Level 2 enabled
[+] Connected to Wayland compositor
[*] Output HDMI-A-1 (Samsung Electric Company C24F390 H4LT305839) selected
[*] OpenGL 4.6 EGL context created
[+] EGL initialized
 (+) Video --vid=1 (*) (vp9 3840x2160 25.000fps)
 (+) Audio --aid=1 --alang=eng (*) (opus 2ch 48000Hz)
[*] Loaded /home/raffaele/Videos/wallpaper.webm
[+] MPV initialized
[*] MPV is ready to render the next frame for HDMI-A-1
AO: [pipewire] 48000Hz stereo 2ch floatp
VO: [libmpv] 3840x2160 yuv420p
[*] MPV is ready to render the next frame for HDMI-A-1
[*] HDMI-A-1 is ready for MPV to render the next frame
[*] HDMI-A-1 is ready for MPV to render the next frame
[*] MPV is ready to render the next frame for HDMI-A-1
AV: 00:00:10 / 00:02:00 (9%) A-V:  0.000 Dropped: 261

[*] Exiting mpvpaper

[GhostNaN]

Screw it! Here reordered_egl_main_3.tar.gz Same thing, but polling removed. This, in theory, should put it basically back to where it was before https://github.com/GhostNaN/mpvpaper/commit/0475df808f302c11c2d69ad1bd32fd2acb9cf2cf If not, IDK know what I'm doing.

[raffaem]

Screw it! Here reordered_egl_main_3.tar.gz Same thing, but polling removed. This, in theory, should put it basically back to where it was before 0475df8 If not, IDK know what I'm doing.

Yes, that works. You did it!

[GhostNaN]

Yes, that works. You did it!

Don't get too excited. It's seems like a combination of the EGL initialization and polling that's breaking mpvpaper. Now it's a matter of putting the EGL initialization back together in a way that supports multiple EGL surfaces. I'm hoping the worst case for Nvidia users is just not having the polling feature.

[GhostNaN]

It's not pretty(code wise) but this might fix the multi-monitor support: hybrid_main.tar.gz I just kept/forced the EGL initialization load order the same and added back multi EGL surfaces.

[raffaem]

Can you at least make a branch?

It's difficult to test like this

[raffaem]

It's not pretty(code wise) but this might fix the multi-monitor support: hybrid_main.tar.gz I just kept/forced the EGL initialization load order the same and added back multi EGL surfaces.

Still working.

What was the problem with EGL initialization order?

[GhostNaN]

Can you at least make a branch?

Sure thing https://github.com/GhostNaN/mpvpaper/tree/nvidia-fix

Still working.

Excellent, we got multi-monitors working now! Now, let's see if I can rework the polling to fix the issue completely.

What was the problem with EGL initialization order?

Honestly, I have no damn clue. This is some GPU driver voodoo magic we got here.

[GhostNaN]

Still working.

Wait, this is with multi monitors right? Like: mpvpaper '*' /path/to/video The coded changes the order and that really only effects the first monitor, the rest is like the old way.

[raffaem]

Still working.

Wait, this is with multi monitors right? Like: mpvpaper '*' /path/to/video The coded changes the order and that really only effects the first monitor, the rest is like the old way.

I have a laptop with an external monitor.

I call my compositor after exporting:

export WLR_DRM_DEVICES=/dev/dri/card0

in order to use only the external monitor, for the compositor to see only the external monitor and not the laptop one, as suggested here.

Given this premise,

./build/mpvpaper -vv -o "--loop=inf" '*' "/home/raffaele/Videos/wallpaper.webm"

works

[GhostNaN]

in order to use only the external monitor, for the compositor to see only the external monitor and not the laptop one, as suggested here.

So you don't have a multi-monitor setup. This is a dead end. There might be a way to create virtual monitors, but I'm not sure how that would work.

I think I also over engineered the solution with the EGL init. I'm hoping the simple fix is just having EGL surface and window created before mpv. Try this commit here https://github.com/GhostNaN/mpvpaper/commit/e21891ad3f3b2ed9fa95ff406036bd5f5b5de25c If that works, just as a sanity check try the following commit here: https://github.com/GhostNaN/mpvpaper/commit/33e03d3938c9a69be13d0d5da1ee5935a81c4032

[raffaem]

Nope, commit 33e03d3938c9a69be13d0d5da1ee5935a81c4032 doesn't work. Says dropped frames.

commit e21891ad3f3b2ed9fa95ff406036bd5f5b5de25c doesn't work either. Doesn't print dropped frames, but doesn't show the video either, just a black screen.

commit 7eb7b7029b861fcc026e5edb14a6ef6073c81ea8 work correctly.

[GhostNaN]

Anymore changes to the code will basically lead us back to this point: https://github.com/GhostNaN/mpvpaper/commit/f65700a3ecc9ecd8ca501e18a807ee18845f9441 And I won't be mainlining https://github.com/GhostNaN/mpvpaper/commit/7eb7b7029b861fcc026e5edb14a6ef6073c81ea8 because I have a feeling it will break more than it fixs for everyone else that doesn't use a Nvidia card. As it stands, for your setup and anyone else using the Nvidia drivers, I would just use mpvpaper from this commit: https://github.com/GhostNaN/mpvpaper/commit/f65700a3ecc9ecd8ca501e18a807ee18845f9441 You won't loose any functionality(besides no multi-monitor support) and it won't hammer your GPU like https://github.com/GhostNaN/mpvpaper/commit/7eb7b7029b861fcc026e5edb14a6ef6073c81ea8 will.

I am uncertain where I can take this issue from here. And I'm still not convinced this is entirely mpvpaper's fault here. Whether this is a Nvidia driver or a MPV issue is still unclear.

I really appreciate the effort you have put in to diagnosing this issue. But unless you have any other suggestions, this is where it ends.

[jd1t25]

Hey just heads up. The branch nvidia-fix doesn't work for me. But https://github.com/GhostNaN/mpvpaper/commit/7eb7b7029b861fcc026e5edb14a6ef6073c81ea8 worked

[GhostNaN]

I'm aware, I will most likely just delete that branch as it was a failure. Just use https://github.com/GhostNaN/mpvpaper/commit/f65700a3ecc9ecd8ca501e18a807ee18845f9441 going forward for now.

[raffaem]

@GhostNaN what about crowdfunding a NVIDIA laptop for you? 🧐

[GhostNaN]

@raffaem Very funny... how about we crowdfund everyone getting an AMD laptop instead?

[raffaem]

@raffaem Very funny... how about we crowdfund everyone getting an AMD laptop instead?

I was not joking