Can't remote input while screen locked.

[jgottfried]

When GNOME screen is locked, I can not wake it up from my phone. Neither keyboard nor mouse control will work, if the GNOME session is locked.

Sometimes when I just need a very quick check up on my PC, I pick up my phone, open KDE Connect, use remote input to move the mouse, the screen wakes up and I can see notifications, progress bars etc. from the other side of the room.

This works when kdeconnectd is running; if necessary I can even type in my password and unlock my user session from my phone.

With GSConnect this does not work. I can control media, I can control volume, but I can not make the screen wake up from sleep. Now this might be a very niche use case and normally I wouldn't question it, this could well be a security feature, maybe GNOME Shell limits extensions while the screen is locked – but it works with kdeconnectd and this is supposed to be a complete implementation, to me this is a regression when using GSConnect. What good is the ability to remote control your PC if you first have to walk over to it and unlock it?

Steps To Reproduce:

1. Open KDE Connect on phone
2. Open "Remote input" and move your finger around, the mouse moves on the desktop.
3. Lock the desktop screen
4. Move your finger on the phone: nothing happens.
5. Turn off GSConnect, start KDEConnect
6. Lock the desktop screen again
7. Move your finger on the phone,
8. the desktop screen should wake up.

Expected behavior

I expect it to behave like KDE Connect, in that it will move the mouse pointer and wake up the system, even when the screen went to sleep and/or the session is locked.

System Details (please complete the following information):

• GSConnect version: 47
  • Installed from: GNOME Extensions Website
• GNOME/Shell version: 41
• Distro/Release: Arch

GSConnect environment (if applicable):

• KDE Connect app version: 1.18.1

I haven't looked into making a command to unlock the session from my phone, that might be a work-around.

[andyholmes]

Thanks for reporting.

Looks like this would require GSConnect to change the SessionMode from restrictive to something else or to modify the restrictive mode variable allowScreencast to true. Whether that's actually a good idea, I couldn't say.

An alternative could be to implement the lock plugin, but since kdeconnect-android doesn't seem to support that plugin yet, that won't really help in this situation.

[Neon-44]

is this still on the radar? can i do something to help resolve this issue?

anyways, making a Command to unlock is NOT a workaround, as the commands don't work either. sadly

[andyholmes]

GSConnect is a community-driven project, so it relies on people contributing the features and fixes they want to see. There are maintainers around to review and help out, but otherwise it's up to community members to keep the project alive.

[Neon-44]

as much as i would have loved to get my hands dirty in some javascript and help fix this problem (despite being 100% unqualified), i fear that we can't fix it at all:

i found out that Gnome disables Extensions for security reasons when the screen is locked.

GsConnect would need an entire rewrite as a stand-alone-application instead of an shell extension.

so i guess unless you guys plan to rewrite the entire thing we can close the issue

(btw @andyholmes , i know that this is a community-driven project, that's why i asked whether i could do something to help you guys resolve the issue ;D)

[andyholmes]

i found out that Gnome disables Extensions for security reasons when the screen is locked.

That's not necessarily true, it's just that GSConnect doesn't request the Session Mode for unlock-dialog.

GsConnect would need an entire rewrite as a stand-alone-application instead of an shell extension.

Actually most of GSConnect is already a standalone application. There are some details in the CONTRIBUTING.md guide about GSConnect's architecture that might help explain how it all fits together.

so i guess unless you guys plan to rewrite the entire thing we can close the issue

Well, I'll leave that up to the original reporter :)

...that's why i asked whether i could do something to help you guys resolve the issue

Yup, my first comment is the description of one way to resolve the issue. I believe there are some extensions that already do something similar, but there also might be other ways to fix it.

[Nocccer]

I know that this worked before and at a certain point it blocked. @andyholmes if you tell me at which file i need to look, i could try to fix this.

[andyholmes]

I think the easiest way is probably to have GSConnect modify the unlock-dialog session mode in GNOME Shell, which can be found in js/ui/sessionMode.js.

Probably the only way that this will pass review is if you add support to GSConnect for the unlock-dialog session mode as described in the Session Mode documentation, then make that change in the enable()/disable() hook in extension.js.

[Nocccer]

Looking through the code, there is a setting in preferences. I have no clue how to enable this? https://github.com/GSConnect/gnome-shell-extension-gsconnect/blob/2039513393da9279206c874abdda16ced427532e/src/preferences/service.js#L231

[Nocccer]

Ok if i understand this correctly, this just stops the service and does not destroys it. What if we just return there instead of stopping it. Should this not keeping GSConnect alive in lock screen?

https://github.com/GSConnect/gnome-shell-extension-gsconnect/pull/1537

Sadly i can't test the nightly build, because my distro (Pop_os 22.04) does not support gnome 43 atm.

[andyholmes]

What if we just return there instead of stopping it. Should this not keeping GSConnect alive in lock screen?

Those are really unrelated things. The this.service object in extension.js is just a D-Bus proxy for the actual service. I know this is a point of confusion for a lot of folks, but most of GSConnect is a standalone service.

The actual extension just adds a menu, a few notification features like quick reply and controls for starting/stopping the service (i.e. start()/stop()).

Sadly i can't test the nightly build, because my distro (Pop_os 22.04) does not support gnome 43 atm.

That behaviour was only changed in GNOME 43, so if you're running GNOME 42 that change shouldn't have affected you. Does GSConnect actually disconnect from your phone when the screen locks, or is it just that remote input stops working?

[Nocccer]

Ok adding this "session-modes": ["user", "unlock-dialog"] in my local installation metadata.json file. GSConnect does show the GSConnect indicator in the locking screen now. Still the input does not work. Seems not a problem with the missing "unlock-dialog".

In both cases my phone still detects my PC and and vice versa. I also can open the remote entry, but if i try to move the mouse nothing happens.

That behaviour was only https://github.com/GSConnect/gnome-shell-extension-gsconnect/pull/1475, so if you're running GNOME 42 that change shouldn't have affected you. Does GSConnect actually disconnect from your phone when the screen locks, or is it just that remote input stops working?

If i installed the latest release or the latest nightly build, the Extensions stops working and i get a error that this extension is incompitable with my current gnome version (42.5).

[andyholmes]

Still the input does not work. Seems not a problem with the missing "unlock-dialog".

There's still some digging to do I guess :slightly_smiling_face:

Here you can see that allowScreencast is false in the restricted session mode, and the unlock-dialog session mode inherits from that mode. Maybe the Allow Lockscreen Remote Desktop extension will solve this problem for you?

[pesader]

I made a quick test, where I added a line of code from Allow Lockscreen Remote Desktop to the enable function of GSConnect, which now looks like this:

function enable() {
    serviceIndicator = new ServiceIndicator();
    Notification.patchGtkNotificationSources();

    // I added this part
    global.backend.get_remote_access_controller().
        inhibit_remote_access = () => {};
}

and it seems to be "working" :)

I say "working" because the GSConnect now takes ages to start functioning when a session is started. The phone is properly connected, but the user is not able to send clipboard, run commands, remote input, etc. After a while, these things start functioning again as expected, but I was unable to figure out why.

[andyholmes]

Hmm, not sure. I suppose if the device was disconnected and reconnected when the lock enables, it might be choking up on the initial onslaught of packets that happens at connect-time. Logs may tell?

[pesader]

The slow down only happened upon login, and I couldn't find a way to trigger it manually (disconnecting and connecting my phone, disabling and enabling the extension, etc). Fortunately, the slow down didn't happen again! No idea what was causing it.

With that out of the day, where would be good place to add these changes? Since the enable() and disable() functions only have two lines each, I assume it's not a great idea to add anything there directly.

I also plan to only enable that feature if the "Keep alive" option is toggled on.

[pesader]

@andyholmes, sorry for tagging you but could you give me some guidance again? 😅

[andyholmes]

With that out of the day, where would be good place to add these changes? Since the enable() and disable() functions only have two lines each, I assume it's not a great idea to add anything there directly.

I would say probably create shell/input.js and add setup/teardown functions like shell/notification.js does. Those can go in enable()/disable() or I guess signal callbacks for the GSetting.

[pesader]

Thank you for the quick reply! I'll give this a shot :)

[ferdnyc]

@pesader Was this actually fixed by #1575? I notice the issue wasn't linked to close with the PR. Let us know if this is working for you with that change merged.

[pesader]

@ferdnyc I forgot to link the issue in my commit message, my bad 🤦🏻‍♂️

But yes, it's working! Matter of fact, last sunday I helped a friend install Fedora on her computer and when I was configuring GSConnect for her, I tested remote input on the lockscreen and it worked as expected :)

[ferdnyc]

Great, thanks! Closing as fixed.