Keavon
commented
2 years ago Our use case is a 2D raster and vector graphics editor application called Graphite which will support desktop and web. For the web version, we are running into limitations of the web API because of the lack of this proposed feature.
I implemented some heuristics to guess based on a combination of event.key and event.code values but it isn't sufficient. We need the ability to query what key will come from hitting each key on the keyboard with each modifier key pressed (including the Option key on Macs, which acts as an alternate symbol key). That is necessary for both input mapping (to detect what relevant symbol the user actually is pressing regardless of modifier keys) and display of the keyboard shortcuts in the UI.
Without this proposed addition to the spec, it is essentially impossible to properly build localized web apps with nontrivial keyboard shortcuts that work outside the US.
For example, a user pressing AltW (OptionW on a Mac) will end up typing an event.code of KeyW and an event.key of Σ. On the US layout, it's fine to use the code, but on a French layout, KeyW should be a Z not a W! On a French layout, there is no way to determine which key the user pressed while holding the Option modifier. This is also true with the Shift modifier for any key that doesn't have a lowercase and uppercase version of the same symbol (where .toLowerCase() would be handy), like the number row that puts numbers and symbols on the same keys. It's impossible to know where a certain symbol is, and whether it's behind a modifier or not, for every keyboard without the proposed API addition.
Our graphics editor application is one example amongst many of a professional-oriented web app that needs full control over keyboard shortcuts. Other examples can include Figma, a hypothetical web app port of Blender and other similar tools... basically all digital content creation tools. That's in addition to the use case presented in the proposal: IDEs.
The current draft states the following non-goals:
I understand that for the currently described user scenario, keypress instructions in games, this is acceptable. There is another scenario that is unsufficiently supported by web APIs: command keybindings in web IDEs. In this other scenario, the three stated non-goals are exactly the missing piece that would allow to handle keybindings for IDE commands properly. This can be explained best with an example.
Example
The Visual Studio Code keybinding for the command "Toggle Line Comment" is
Ctrl+/on Windows andCmd+/on Mac with US keyboard layout. On many other keyboard layouts, the/character does not have its own key, so the keybinding has to be changed to something else. For German keyboards, VS Code has solved it by usingCtrl+#on Windows andCmd+Shift+7on Mac (Shift+7produces the character/on German keyboards).VS Code is an Electron application and uses the native-keymap package to determine the keyboard layout (see https://github.com/Microsoft/vscode/issues/713). Since that is an OS-native library, it cannot be used in a web application.
Rationale
If we would have a web API that allows us
a web IDE could identify the complete keyboard layout and thus adapt the configured keybindings to the user's keyboard. This is exactly the kind of information that is provided by the native-keymap package for VS Code.
The inverse mapping, that is the ability
would be useful, too, in that context, though it could also be derived from the mapping above and thus does not add any new information.
In case a web IDE would like to display the detected keyboard layout to the user,
would be necessary.
In summary, web IDEs need some means to query the mappings from
codevalues tokeyvalues for all combinations of theShiftandAltmodifiers. The other extensions can be useful, but can be regarded as optional.Proposal
The current proposal of the Keyboard interface could be extended so the
getLayoutMap()function takes an optional first argument with the structure{ shiftKey: boolean, altKey: boolean }. If such an argument is given, it returns the KeyboardLayoutMap that considers those modifiers, otherwise it behaves as described in the current proposal.Background Information
As a committer of Eclipse Theia, my main motivation is to improve support for international keyboard layouts in that web IDE. Theia is currently used in
Of course other web IDEs could also greatly benefit from better keyboard layout support in the web API, for example
Web IDEs are emerging very quickly, which shows that there is a high demand for them. However, the web APIs are not prepared for this kind of user scenario when it comes to keyboard events.
Privacy Considerations
The impact on users' privacy is basically the same as for the currently proposed API (see #8). Of course adding modifiers to the keyboard layout information increases the fingerprint in terms of the amount of data, but I assume it does not change much about the identifiability of a certain keyboard because most keyboards should be already fully identified by their character layout without modifiers. This assumption needs to be verified, though.
How to Proceed?
I am submitting this issue because the current keyboard-map proposal is already quite close to the needs of web IDEs. The missing piece are the modifier keys.
If you think what I describe would be better put into a separate draft, I'm fine with that. My intention is to start a discussion on this so eventually we can improve the user experience for web IDEs.