Support of different keyboard layouts

[skatromb]

Is there an existing issue for this?

• [X] I have searched the existing issues
• [X] I have checked the FAQ
• [X] I understand this form is not for requesting support for new browsers or apps

Current Behaviour

Hotkeys for different browsers are bind to symbols. Since that, if you change keyboard layout (for example, to Russian), there would be another symbols on the same buttons of keyboard — hotkeys wouldn't work anymore.

Expected Behaviour

Hotkeys should be bound to keys of keyboard — no matter, which keyboard layout is currently in work. So that would lead to situation, when you can use the same button with any keyboard layout and hotkeys would work as usual.

Steps To Reproduce

1. Set keyboard layout as US
2. Set hotkey to Safari as S
3. Switch keyboard layout to Russian
4. Click the link to open the Browserosaurus window
5. Try S button on keyboard (which would be symbol Ы)
6. Hotkey is not working

Browserosaurus version

16.0.7

macOS version

12.1

CPU Architecture

ARM

Anything else?

No response

[will-stone]

Hi @skatromb Firstly, thanks for the coffee 🙂

This is actually how B used to work, but as I was previously hardcoding letters it meant that the result was the opposite on different keyboards and the moved "S" would not work. Looks like I presumed this would be useful one day: https://github.com/will-stone/browserosaurus/blob/master/src/renderers/picker/components/hooks/use-keyboard-events.ts#L20 Now people can set their own hotkeys, we can use what I've called "physical key". I'd need to do some research but looks like a fairly straight forward swap-out. Happy to accept a PR if you know what needs changing in here. Otherwise I'll get to this soon 🙂

[skatromb]

Thank you for quick response, @will-stone!

Yeah, I think use of "physical keys" should work fine for this! What is about PR — unfortunately, I am completely incompetent in JS (so I'll wait until you will have your hands on that :)

[will-stone]

No worries. I like to ask just in case you'd like to contribute with code 🙂 I've just done a release which you can read about here: https://github.com/will-stone/browserosaurus/releases/tag/v16.1.0 But I'll get to this when I can.

[will-stone]

I've had a think about this and it's unfortunately a bit subjective.

Let's say I have Firefox bound to the f key. On my first keyboard layout I can press the f key:

And the picker shows f as the key to press:

Now let's say that I have a second layout where the f and p keys are swapped.

If I touch type, I know that Firefox is still bound to f so I should press the key where f is, and the picker correctly displays this:

Whereas if we make the previously mentioned adjustment of using what I called the "physical keys", we now need to press p:

But the picker would still show f, which could be confusing, is this the physical "f" or the virtual "f"?

So we have two schools-of-thought:

1. Hotkeys should be the same character, no matter where the character is moved around the keyboard, for touch typists. [current behaviour].
2. Hotkeys should be the same physical key, no matter what key is typed when pressing that key, for muscle memory [proposed behaviour].

Then if using 2, does the picker show:

a. The physical key b. The virtual key that is on the originally set physical key

It's a tricky one, and also very hard for me to sympathise with; I only use one keyboard layout. And because I only use one layout, it's incredibly hard for me to justify adding configuration that will require development, testing, and maintenance, for something I will never use.

Let me know your thoughts.

[skatromb]

Hello, @will-stone!

First, thank you for such deep dive in such "simple" feature. Second, thanks for such clear description of your thoughts. Hope, mine will be the same for you.

I think, picker should react and show always EN layout keys (what we call "physical keys" here). Rationale: that is common practice in both MacOS and multi-language apps. I've tried to switch my MacOS language to Russian to proof it, here are my findings:

1. Even in the most edge case, with both RU system language and RU keyboard layout, you have hotkeys legend in menu, written with EN layout in MacOS

   

2. The same situation is in the settings of MacOS hotkeys with RU system language and RU keyboard layout:

   

3. I've tried JetBrains's PyCharm to set hotkey for some action with RU system language and RU keyboard layout (app's language was still EN, it have no RU localisation), and again, EN keys were shown:

   

All it applies to situation, when you have EN system language with any (EN or RU) keyboard layout.

[will-stone]

Thank you for this, that's a strong argument, and is certainly good to know.

I would say then, a good compromise is this:

Use the physical key to set the hotkey, but display what that physical key maps to? So in my example above, it's 2 and b. This allows people who've moved f to still show f even though the physical key is now p.

It seems like this will essentially fix a bug, as I've got it currently set that unless the virtual key is in the range of a-z0-9 it won't trigger a hotkey. Which presumably means your RU layout doesn't work? Or maybe it currently breaks Chinese / Japanese layouts etc.

[will-stone]

Okay, I'm getting close to the above solution. However, like the jump from v15 to v16, this will unfortunately mean all hotkeys will get reset again (hopefully for the last time). This is because the physical key code is a different format to the virtual key code. e.g.

Physical key: KeyA Virtual key: a

Without a migration step, you can see these aren't compatible. And to be honest, for the sake of entering the hotkeys again, I don't see the migration step worth it.

See keycode.info for a demo of the information that is available in JavaScript.

[skatromb]

Sorry for delayed answer, was all on holidays preparations :)

would say then, a good compromise is this:

Use the physical key to set the hotkey, but display what that physical key maps to? So in my example above, it's 2 and b. This allows people who've moved f to still show f even though the physical key is now p.

First my question, would it be only showing the virtual key? Because if it will also react to the virtual key, it won't fix the issue on a Russian layout.

Second, why do you want to implement such mapping? My only answer to this is if someone uses different EN layout such as Dvorak, for example. Is that the case that you want to support?

What I realised, is that such an layouts could be a problem. For example, in Dvorak layout

If you set Dvorak layout and try to set hotkey as a Cmd + S (I mean, physical S, which in Dvorak maps to O), you will see in result in Mac hotkeys Cmd + O. Which is, as I understood, more close to what you explained. Here is the video-proof: https://www.icloud.com/iclouddrive/07aO6Szh1or_BO2kqzRDKiaNg#Screen_Recording_2022-01-01_at_21.03

My only strong will here, is that I will have ability to press exact physical key on both standard Russian and standard English layouts. All other things, like what key would be shown — are completely non-mandatory)

[will-stone]

My only strong will here, is that I will have ability to press exact physical key on both standard Russian and standard English layouts. All other things, like what key would be shown — are completely non-mandatory)

Yes, the updated functionality will be: display virtual key, hotkey bound to physical key.

[skatromb]

Super! Anticipating the new release :)

[ivan-fishbrain]

Tested today, works nice! On both RU and EN layouts shows EN keys, which is fine. 🍻

[will-stone]

RU and EN layouts shows EN keys

Oh, I would have thought that it should display the key in Russian 🤔 Maybe a limitation of JS, or a bug in the code somewhere. I'll reopen and check when I get a moment...

[ivan-fishbrain]

You can pass it as it is currently, as for me, it is best solution) But of course do that as you want!

[will-stone]

I think this is a limitation of JS, as we know the keycode map is working due to the keys updating on change to Dvorak etc.

[saneef]

I'm using Browserosaurus v19.1.0. The keyboard shortcuts changes across different keyboard layouts. :'(

US	Colemak

Isn't the expectation to have the same shortcut keys across the layouts?

[will-stone]

Possibly 🤷‍♂️ Please send a PR if it’s not behaving as expected. Thanks 😊

[saneef]

Today, I tried to find what would be a better approach. It seems, relying on either event.key or event.code doesn't seem to work for different keyboard layouts. Here I've made a table of physical keys pressed using three different layouts: ABC (US QWERTY), Colemak, and Russian.

The English characters are not available on Russian (and other non-English) keyboard layouts. So, when relying on event.key, the Russian layout users may not be able to type the character (hot key) which was set using an English layouts like QWERTY, Dvorak or Colemak.

In the case when we rely on event.code (physical keys) the users of non-QWERTY layouts like Dvorak or Colemak the hotkey is tied to physical key which is not ideal for them. Example, a hotkey set to s in QWERTY will become r in Colemak.

I played with Visual Studio Code (VS Code), an Electron-based app like Browserosaurus, and went through their code, to understand how they handle this problem.

This is how the keyboard shortcuts for moving the cursor to next line, ctrl+n and previous line, ctrl-p work in VS Code.

1. QWERTY: Work like described above.
2. Russian: The n and p are mapped to the physical location on QWERTY keyboard. (similar to relying on event.code)
3. Colemak: The n and p are mapped to Colemak n and p keys. (similar to relying on event.key)

The code related to key codes shows that they use event.key if the code is within ASCII range. Otherwise, relies on physical key (event.code).

Does this approach make sense? When I have some time to spare, I'll try for a PR with fix.

Foremost, this problem is faced by a set of people who regularly switch between layouts. I don't know how big is this subset. Unfortunately, I'm one of them. 😄