search

YaLTeR / niri

A scrollable-tiling Wayland compositor.
https://matrix.to/#/#niri:matrix.org
GNU General Public License v3.0
4.12k stars 123 forks source link

Typing break monitor doesn't quite recognize idle time properly with niri #660

Open jjramsey opened 2 months ago

jjramsey commented 2 months ago

This issue reflects a problem described here: https://github.com/rcaelers/workrave/issues/565#issuecomment-2241148911

On LabWC, Hyprland, and the new COSMIC desktop alpha, installing xdg-desktop-portal-gtk is enough to get time watching video on Firefox (without keyboard or mouse input) to be recognized as idle time in the Workrave typing break monitor, probably because Firefox uses the dbus org.freedesktop.impl.portal.Inhibit interface from the GTK portal (rather than Wayland's idle-inhibit protocol, which works poorly with typing break monitors).

However, what works with LabWC, Hyprland, and COSMIC doesn't work with Niri, and I don't know why.

System Information

YaLTeR commented 2 months ago

Niri does implement the freedesktop Inhibit interface and Firefox video watching is supposed to trigger it. You can test it by starting swayidle timeout 1 'echo hi', then starting a YouTube video on Firefox. For me it doesn't print while the video is playing, meaning that idle is successfully inhibited.

jjramsey commented 2 months ago

I think you misunderstand. Firefox will try (at least?) two methods of idle inhibition: first, the one involving the Freedesktop org.freedesktop.impl.portal.Inhibit interface -- which doesn't interfere with typing break monitors like Workrave, while still preventing idle monitors from timing out -- and second, the Wayland idle inhibit protocol extension -- which is a lot more coarse-grained and prevents typing break monitors from recognizing intervals without user input as idle.

Your test can't tell the difference between the two methods of idle inhibition.

Something in the setup for Niri's seems to prevent the org.freedesktop.impl.portal.Inhibit interface from becoming active, causing a fallback to the more coarse-grained Wayland idle-inhibit protocol.

YaLTeR commented 2 months ago

I looked into this when implementing inhibiting, and Firefox pretty much cannot ever use the Wayland inhibit because the portal Inhibit never fails. I believe I opened some issues in the portals about this, but they always returned success. This was why I even implemented the freedesktop inhibiting in niri in the first place.

You can actually tell these apart. The Wayland inhibiting is only active when the inhibiting window is on screen. If I do the swayidle test with a Firefox YouTube video, then switch to a different workspace, swayidle still does not print anything, which indicates that Firefox uses the freedesktop inhibit on my system.

jjramsey commented 2 months ago

Then why is Workrave acting as if the Wayland inhibit protocol is being used?

There's a pretty straightforward check:

(ETA: I did try switching to another workspace, and Workrave's timer did still count down, so I guess Niri's org.freedesktop.impl.portal.Inhibit implementation is being used instead of the one from the GTK portal. That would indicate that Niri's implementation has the same weakness as the Wayland inhibit protocol of failing to deal with the case where the computer is not idle but the user is. The GTK and KDE implementations of the org.freedesktop.impl.portal.Inhibit interface do not have this problem.)

YaLTeR commented 2 months ago

swayidle should still treat this time as not idle, because it's receiving signals from the org.freedesktop.impl.portal.Inhibit

I'm fairly sure that swayidle only receives idle events from the ext-idle-notify-v1 Wayland protocol, i.e. exactly what niri tells it.

Niri's implementation has the same weakness as the Wayland inhibit protocol of failing to deal with the case where the computer is not idle but the user is.

The most common use for the idle protocol is to power off the screens after some time of inactivity, and the most common use for inhibiting idle is to prevent this powering off from happening as you're watching a movie. So arguably the fault is with the program that assumes it can use this idle indicator for user inactivity.

jjramsey commented 2 months ago

I'm fairly sure that swayidle only receives idle events from the ext-idle-notify-v1 Wayland protocol, i.e. exactly what niri tells it.

IIRC, swayidle also listens for D-Bus signals that tell it not to trigger its idle timeouts. (I think this works via systemd.) I already tested this in Hyprland. If I run swayidle timeout 1 'echo hi', it stops repeating 'hi' when playing video in Firefox even though Workrave still recognizes that the user is idle.

The most common use for the idle protocol is to power off the screens after some time of inactivity, and the most common use for inhibiting idle is to prevent this powering off from happening as you're watching a movie. So arguably the fault is with the program that assumes it can use this idle indicator for user inactivity.

If you look at the discussion at https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/29, you can see that RSI tracking (i.e., tracking user activity to help deal with Repetitive Stress Injury to hands and wrists) is part of the use case for the ext-idle-notify protocol. So no, it's not an abuse of the protocol, but rather an expected (though niche) use.

There already is a way to keep the computer from powering off without falsely indicating that the user is typing or using the mouse. That's what the org.freedesktop.impl.portal.Inhibit D-Bus interface is good for. The one from the GTK portal works just fine in LabWC, Hyprland, and COSMIC, and the one from the KDE portal works just fine in, well, KDE. Niri is the outlier here.

YaLTeR commented 2 months ago

I'm not sure what you're suggesting. Niri does not inhibit idle through the dbus interface or "interfere" with it. It listens to it to relay the inhibiting to the Wayland idle notify protocol.

jjramsey commented 2 months ago

I'm not sure what you're suggesting. Niri does not inhibit idle through the dbus interface or "interfere" with it. It listens to it to relay the inhibiting to the Wayland idle notify protocol.

So it isn't trying to implement org.freedesktop.impl.portal.Inhibit at all, and just uses whatever implementation comes from an XDG portal?

(I thought you indicated that you did implement it, but I guess I misunderstood.)

YaLTeR commented 2 months ago

Niri doesn't implement any portals. What it does implement is an org.freedesktop.ScreenSaver inhibit interface https://github.com/YaLTeR/niri/blob/main/src%2Fdbus%2Ffreedesktop_screensaver.rs#L23 which the gtk inhibit portal impl calls, if I'm not mistaken.

jjramsey commented 2 months ago

Gotcha.

If I wanted to test a version of the Niri code without that org.freedesktop.ScreenSaver interface implementation, how would I go about it? Loosely speaking, what would I need to comment out without breaking anything I didn't want to break?

YaLTeR commented 2 months ago

Should be fine to just comment this:

https://github.com/YaLTeR/niri/blob/96083847fb8776ca4f0ed613148e2e6540690cb3/src/dbus/mod.rs#L56-L57

jjramsey commented 2 months ago

That did it. I tried a version of Niri with those lines commented out, and I got the behavior with Workrave that I get with those other compositors that I mentioned.

ETA: After I got the behavior of Niri to be how I wanted it, I got a bit fancier. In niri/niri-config/src/lib.rs, I added

#[knuffel(child, default)]
    pub prefer_no_screensaver_impl: bool,

to the Config struct, and in niri/src/dbus/mod.rs, I now have

            if !(config.prefer_no_screensaver_impl) {
                let screen_saver = ScreenSaver::new(niri.is_fdo_idle_inhibited.clone());
                dbus.conn_screen_saver = try_start(screen_saver);
            }

My modified Niri now does what I want it to do if I have prefer-no-screensaver-impl in the config file. I don't know if I'd really want the option to be called prefer-no-screensaver-impl, but it's at least a proof-of-concept for a reasonable solution.

YaLTeR commented 2 months ago

Well, the unfortunate thing is that without it, flatpak Firefox has no way to stop swayidle from turning off the monitors. Due to the aforementioned bugs in the portals that always return success, so Firefox never falls back to the Wayland inhibiting. This goes for any flatpak app that only knows about the inhibit portal, too.

jjramsey commented 2 months ago

Yes, but the needs of those who have RSI and use typing monitors to keep it in check are worth considering as well, and my proposed fix with the prefer-no-screensaver-impl can help accommodate them.

Consider it a workaround for the messy state of idle monitoring on Wayland, which kinda sorta distinguishes between the user being not idle (by providing input) and the computer being not idle (because it's playing video or whatnot), except when it doesn't. Basically, we have something like this:

Arguably, in a more ideal world, we'd have the Wayland compositor take the responsibility for measuring user activity and the D-Bus protocols to indicate that the computer is doing something that shouldn't be interrupted via suspension, screen locking, etc. In the current world, we almost have this, but the Wayland idle-inhibit protocol complicates the former, and dodgy implementations of the org.freedesktop.impl.portal.Inhibit interface complicate the latter.

My proposed fix with the prefer-no-screensaver-impl option would help navigate this mess somewhat -- though it would have its tradeoffs -- at least until the current state of idle monitoring gets better.

ETA: I'm up for creating a pull request with the proposed fix, with perhaps a better name for the option than prefer-no-screensaver-impl.

YaLTeR commented 2 months ago

I can't promise I'll merge that because really it is a hacky workaround that just happens to work for your particular scenario. Had Firefox successfully used the Wayland inhibit instead, you would end up with the exact same problem where idle time watching a video is not counted as idle time. Or with any other client that uses the Wayland inhibit.

You may want to just keep running the patched niri version for this, it's not like this delete 2 lines patch is gonna break.

I can see how a way for monitoring the user activity is useful. It would not even be hard to implement, in fact it would be strictly easier to implement (the same as the current idle-notify but with no inhibiting). If there's another D-Bus interface or protocol for this, we can implement that. Or we could create one (but then the target app also needs to know about it).

Actually, it seems that Workrave can use a Mutter-specific D-Bus interface: https://github.com/rcaelers/workrave/blob/main/libs/input-monitor/src/unix/MutterInputMonitor.cc I don't know if it handles inhibiting better, maybe it does.

jjramsey commented 2 months ago

I can't promise I'll merge that because really it is a hacky workaround that just happens to work for your particular scenario.

I agree that it would be a very imperfect workaround for the reasons that you mentioned, but it would at least allow Niri to be as accommodating to Workrave (and similar typing break monitors) as the rest of the Wayland compositors that implement ext-idle-notify. I would consider it a simplistic, relatively noninvasive short-term solution that lets other Workrave users besides me use Niri.

Longer term solutions would unfortunately likely require changes to Wayland protocols. 😬 Perhaps, if you are willing to go down a rabbit hole, you could modify Niri to provide proof-of-concept implementations of what the updated protocols could look like. I can conceive of a few possible ideas, though how wise or feasible they'd be is up in the air:

Again, I don't know how good or feasible any of these ideas would be, and I'm not entirely comfortable with proposing ideas that I wouldn't know how to implement. Take them for whatever they're worth.

TelarinT commented 1 month ago

Due to the aforementioned bugs in the portals that always return success, so Firefox never falls back to the Wayland inhibiting. This goes for any flatpak app that only knows about the inhibit portal, too.

There is a way to force the Inhibit portal to fail - by using org.freedesktop.impl.portal.Inhibit=none; in niri-portals.conf. I tested by adding the line and checking dbus activity using Bustle, and by calling swayidle timeout 1 'echo hello world'. Bustle did throw an error for org.freedesktop.impl.portal.Inhibit and swayidle is able to print only if (flatpak) firefox is in a different window

If this is the case could we use this method to force (flatpak) firefox to use the wayland protocol?

YaLTeR commented 1 month ago

If I recall correctly, this error from the impl is not propagated to the application. The portal call (without .impl.) still returns success.

TelarinT commented 1 month ago

Screenshot from 2024-10-17 10-22-56 This is the screenshot from Bustle when the Inhibit portal was disabled in portals.conf. You can see that the portal threw an error (without the Impl, my bad on saying that org.freedesktop.impl.portal.Inhibit threw an error). You can also see when the video state changes (those mpris2 calls), it stops calling the Inhibit portal entirely.

Maybe this is because I am using fedora flatpak instead of flathub (flathub doesn't have aarch64 builds), but they should still use xdg-portals.

I think when you were implementing the ScreenSaver interface it was an issue but it seems to be fixed in https://github.com/flatpak/xdg-desktop-portal/pull/1255 Edit: gtk inhibit portal is still broken, we are just bypassing it by setting it to none

YaLTeR commented 1 month ago

Oh yeah, I remember bumping into that issue. That must've been it for this particular problem.