Lock screen behavior may be surprising

[wolfmanstout]

Creating this bug for discussion. I was surprised to see that I could still send commands to Talon even when the screen is locked, and that the GUI layer still shows. In retrospect this is potentially useful behavior, because it allows users to interact with the lock screen. On the other hand, because currently the only visual indicator that Talon is awake is in the system tray, it was a surprise to find this. It also means that if the user did put talon to sleep, there is no visual indicator. Perhaps this behavior (working on the lockscreen) could be off by default and the user could configure. Alternatively, this is less surprising when using talon_hud, because that does have an indicator that is visible when the screen is locked. If that becomes the default GUI, then maybe the current behavior is a reasonable default.

[ziemus]

I came across this yesterday when I locked my laptop (win10) and could still issue any command. This is a major security bug. Imagine leaving your computer in public or at a workplace locked with a password, yet available for everyone to use.

By the way, I know how I might sound but I am not blaming aegis for this. Bugs happen. I am just surprised that almost nobody's talking about this.

[lunixbochs]

My understanding is that the simulated keystrokes interact with the lockscreen, not the locked desktop. Is that incorrect?

[lunixbochs]

On Windows 11, for me, locking the screen focuses "LockApp.exe" for the fully locked portion, then "Windows Explorer" for the unlock portion (from what I can tell, it's not a file explorer window, it's a generic shell running on the secure desktop and not your actual desktop).

I tried a few things, including "focus appname", and keystrokes / interaction didn't occur behind the lock screen.

My severity rating based on that test would be "informational", or "low". To classify it any higher we'd need to show some kind of tangible security impact to issuing commands in this environment.

I'm not aware of any commands in knausj_talon that will perform a dangerous action in a completely noninteractive way. You could bind one, such as "delete all my files", but that would likely be a bad idea independently of the lock screen behavior.

[ziemus]

My understanding is that the simulated keystrokes interact with the lockscreen, not the locked desktop. Is that incorrect?

Talon interacts with both.

When I lock my desktop and I say, for example, 'launch chrome', I will launch my browser. As long as I don't trigger the login prompt. The same goes for: talon wake, talon sleep, control mouse and probably other commands, too. I think that as long as you don't simulate pressing keys, you can execute commands in lock screen.

If you issue enter, slap, touch, pop, say or any other command that presses keys/other characters (in windows) the login form will appear. You cannot enter anything into the password/pin field so you cannot log in. Nor can you click the ok button or press it by saying enter (that logs in if the windows user has no password set). But you can still issue other commands even then. They will execute immediately after you've logged in (from what I can gather) as they do appear in the command history.

[lunixbochs]

When I lock my desktop and I say, for example, 'launch chrome', I will launch my browser. As long as I don't trigger the login prompt. The same goes for: talon wake, talon sleep, control mouse and probably other commands, too. I think that as long as you don't simulate pressing keys, you can execute commands in lock screen.

I was specific about simulated keystrokes, because that is the main thing I know to be dangerous. Toggling Control Mouse isn't a security risk. launch chrome is getting warmer, but is still conditional on having something more dangerous to launch.

They will execute immediately after you've logged in (from what I can gather) as they do appear in the command history.

I don't think that's correct, they execute while you are still on the login screen and in most cases fail to do anything (e.g. your keystrokes don't go anywhere, as no app is focused)

I will look into blocking commands on the lock screen. The main reason I didn't when this issue was originally raised is it may be an accessibility regression for some people. There are Talon users who physically do not have the use of their hands at all, so I want to minimize the areas of their computer they can't interact with. Right now they can feasibly exit the lock screen if their account has no password required, and blocking commands would prevent that.

This is also unfortunately difficult to do correctly on Linux, as there isn't just one standard lock screen for me to detect.

[ziemus]

I do understand your accessibility concerns, they are valid. But if by exit the lock screen you mean effectively log in then that is not possible, at least on windows (please, correct me if I'm wrong). Talon does not interact with the login form that appears on lock screen. So as of right now, there is no real accessibility drawback when it comes to disabling commands on lock screen.

[lunixbochs]

It looks like we can currently interact with the first screen (LockApp.exe) but not the second one (served by Windows Explorer). I expect the password form itself is a secure desktop so we would need a SYSTEM service or keyboard driver to interact with it (I do want that, so we can also interact with UAC dialogs on the secure desktop, but that's a problem for later).

If the user doesn't have a password, I suspect we won't end up on the secure desktop for password entry so Talon may actually be able to dismiss the lock screen for them.

Regardless, I'll likely start by preventing TalonScript from calling actions on the lock screen. This will block the typical path for hotkeys and voice commands to have any side effects. It won't block pop/hiss handlers in Python, which I consider to be minor for now as most people only use that for clicking anyway. The filter will work for pop/hiss mappings once I move their bindings into .talon files, which I do plan to do.

There may be a middle ground where I allow voice commands on the lock screen but filter a list of known safe actions - so they can basically only press keys, move the mouse, and click, and only if a known window of the lock screen / login form is focused. That could allow login but not anything more dangerous.

Please let me know if you encounter anything that changes the security impact of this. I do think it's important for Talon to have a good answer to this, but am not convinced there are any commands currently bound that turn it into a high severity issue for general users.

[ziemus]

If the user doesn't have a password, I suspect we won't end up on the secure desktop for password entry so Talon may actually be able to dismiss the lock screen for them.

If a windows user doesn't have a password set, they still need to click a 'sign in' button (with mouse button, space or enter) after dismissing the lock screen. The button is not accessible to talon, just like the password field. I am not sure how to test this but I'm pretty sure that this is a secure desktop, too.

[lunixbochs]

In beta 0.2.0-546, any TalonScript binding (commands, hotkeys, face expressions, parrot noises), is blocked on the lock screen. Speech phrase callbacks are also suppressed, which prevents Save Recordings, Show Subtitles, and Command History from running.

[lunixbochs]

Talon 0.3.1 blocks inputs such as voice commands and hotkeys on the lock screen.

[davidsgrogan]

I frequently used talon to input my password on the mac lock screen. "talon wake" followed by "<formatter> <some words>". It was great and I miss it!

If the middle ground doesn't enable formatters, but still allows more primitive commands, e.g. "word", that'd still be great.

[AndreasArvidsson]

I would also like to be able to unlock my computer using talon. Wouldn't it be possible to use a mode on the lock screen? ie disable command mode and enable something like lock_screen. That gives the user control over what is possible to run on the lock screen and not.

mode: look_screen
-
<user.any_alphanumeric_key>: "{any_alphanumeric_key}"

[lunixbochs]

That still may result in users enabling unsafe behavior on the Lock Screen, which was already the concern. Also it opens lots of other questions, like "what if a lock_screen command tries to enable another mode?" And "how do I disable Lock Screen commands while on the Lock Screen?" ("Talon sleep" equivalent, if you have a forced mode... how do you turn it on/off? What does this mean for commands in the "all" mode, or "not sleep" mode?) It's probably so complex I'd rather not do it that way. I think my proposed solution (only allow keyboard/mouse commands) would be easy to reason about and much harder to break.

Also Andreas, aren't you on Windows? As per the previous conversation, I'm pretty sure Lock Screen input only worked on Mac anyway. Implementing it on Windows or Linux is a feature request that's probably outside the scope of this issue.

[AndreasArvidsson]

Sure but it's the user's own computer if they want to degrade their own security shouldn't it be their choice? But you're probably right that it creates more complexity. I just love the freedom to set up my own computer the way I like.

Currently I use windows and linux. At home I don't use the lock screen so it's no problem there, but I recently got back to working at the office and it would be nice to be able to use talon in the lock screen there. I have no idea if talon used to work on the lock screen since I only recently started having that need.

[lunixbochs]

You shouldn't need the full power of talon to get past the Lock Screen, and accidentally having the full power and grammar of talon enabled there is a security trap. Adding a bunch of complexity on the talon side will make it both a security trap and confusing to use correctly. My proposed solution seems like the simplest way to make it very difficult for the user to mess up, without preventing the user from interacting with the login screen.

[AndreasArvidsson]

That could probably work fine. I guess that won't support actions in the user space? eg formatters, noise handlers, etc

[ziemus]

The issue still persists on 0.3.1 but on the startup screen on windows. I have 'start on login' enabled in talon but it starts even before I log in. So I can issue commands before I log in, then log in and have the logged in user's talon carry out commands that I issued in the startup screen. The noise handles still work in both the lock screen and the startup screen. I don't know about the parrot integration as I do not have Talon beta but somebody should check that, too.

[lunixbochs]

@ziemus interesting, is that just the first login? If you login, then re-lock the screen, does the re-lock correctly suppress commands?

I didn't know windows would start things before you actually logged in. The official recommendation from Microsoft was to detect screen lock by detecting the transition between locked/unlocked, but that definitely won't work if you manage to start Talon before logging in.

[ziemus]

@lunixbochs If I relock the screen or put my computer to sleep then talon only executes noise bindings and suppresses voice commands. If I sign out then talon quits and doesn't restart on lock screen. The launch on login is triggered only on the first lock screen after turning the computer on. And only then talon carries out voice commands along with noise handles.

[lunixbochs]

I'm actually a little confused by this, I didn't think Windows would start your user session on first boot until you logged in, but maybe it does this on single user machines to make it seem like the boot was faster.

So I can issue commands before I log in, then log in and have the logged in user's talon carry out commands that I issued in the startup screen.

Do you mean the commands run after you login, or that they run while on the lock screen? If you enable Save Recordings, you can look at the file creation times for your recordings to determine when exactly the command was recognized/executed. When do the subtitles show up? Can you record the time when your computer starts, and the time when you log in, and compare that to Talon's startup time in View Log?

What's your exact windows version? Did you use Talon's "Start on Login" menu item, or some other method to make Talon start on boot?

Thanks

[ziemus]

I'm actually a little confused by this, I didn't think Windows would start your user session on first boot until you logged in, but maybe it does this on single user machines to make it seem like the boot was faster.

Maybe it is a fast boot thing but I should clarify that I have 2 user accounts on my computer, both of which have talon installed and both of them have start on login enabled. The one that shows first on the lock screen is the one I tested this all with, and this is the one which executes the commands. I haven't yet tried logging into the other account to see what happens on the lock screen after I restart the computer. I'll get back to you when I do.

Do you mean the commands run after you login, or that they run while on the lock screen? If you enable Save Recordings, you can look at the file creation times for your recordings to determine when exactly the command was recognized/executed. When do the subtitles show up? Can you record the time when your computer starts, and the time when you log in, and compare that to Talon's startup time in View Log?

The recording creation time match the time before the login. I waited a few minutes on lock screen to make sure. The logs show that talon starts after the lock screen shows up. Not immediately though. In the last case, lock screen showed up at 12:00:49, talon started logging at 12:01:18. The subtitles show up on the lock screen, both the out-of-the-box talon subtitles and in talon hud event log.

What's your exact windows version?

64bit Edition Windows 10 Home Version 21H2 OS build 19044.1889 Experience Windows Feature Experience Pack 120.2212.4180.0

Did you use Talon's "Start on Login" menu item, or some other method to make Talon start on boot?

Start on login, the talon menu item.

[lunixbochs]

early start on windows should be handled in latest beta