[mediaqueries-5] Detect physical keyboard (keyboard capable of shortcuts?)

[birtles]

From stackoverflow: https://stackoverflow.com/questions/36976117/css-media-query-for-whether-a-physical-keyboard-is-attached

Use case: I want show keyboard shortcut labels in my menu / tooltips but only when the user has a physical keyboard.

Rationale (taken directly from the above SO issue):

• nobody would use a soft-keyboard this way even if they could
• the visual display of the keyboard shortcut takes up valuable real-estate
• it's perhaps confusing, or at least inelegant, to display keyboard shortcuts in a keyboardless context

Apparently some people are doing horrible hacks like using @media (hover: hover) and (pointer: fine) { ... } to detect if there is a mouse/trackpad and assuming that means there is a keyboard too.

[tabatkins]

That seems like a pretty reasonable use-case, and I know we thought about whether to expose more input devices.

Thoughts, @frivoal? I'm inclined to add this.

[frivoal]

I've long thought we should be adding something along these lines, without being exactly sure about what we need to add:

• do we need to distinguish between keyboards with and without a numpad?
• do we need to distinguish between computer keyboards and feature phone keyboard that only have a numpad and a handful of extra buttons? How do you classify less-than-full-but-more-than phone things like this: https://web.archive.org/web/20180806062750/https://icdn4.digitaltrends.com/image/blackberry-passport-simple-720x720.jpg?ver=1.jpg
• what about things like gamepads that aren't keyboards but do have buttons? If gamepads aren't keyboards, what is this: https://images.techhive.com/images/idge/imported/imageapi/2014/06/27/21/igrip-100351419-gallery.idge.jpg ? based on what criteria?
• distinguish between keyboard with or without a tab key? with or without arrow keys?
• This probably counts as a physical keyboard rather than on virtual one, on account of not being on screen. But are we sure? That probably depends on the use cases: https://images.techhive.com/images/idge/imported/imageapi/2014/06/27/21/virtual_keyboard-100351425-gallery.idge.jpg
• Do we distinguish between keyboard layouts? Maybe we don't need to distinguish between qwerty and azerty layout since they can type the same characters (mostly), but we do need to distinguish between keyboards that can type ascii letters vs those that can only type other stuff, like https://www.dsi-keyboards.com/wp-content/uploads/2015/03/KA-Thai-20868.jpg
• Do chorded keyboards fall into any of the categories we're otherwise deciding on? if not, does it mean we got them wrong, or that we don't care about such devices?
• in the case of software keyboard, do we distinguish between those that reduce the viewport size when the pop up, and those that get overlaid over the viewport?
• Do we need the primary / non primary input distinction?

I don't have good answers to these questions.

To be clear, I don't mean that we need to have a super rich set of media queries to distinguish between everything. I do mean that the reality of input devices is extremely varied, and that how much we want to expose, and along which axises is not an obvious question. The answers will depend on which set of use cases we want to address. We have a few identified use cases but overall we haven't yet tried to do a broad survey of what people were trying to achieve with these.

Related reading:

• https://github.com/w3c/csswg-drafts/issues/621
  • https://bugzilla.mozilla.org/show_bug.cgi?id=787444
  • https://lists.w3.org/Archives/Public/www-style/2015Apr/0052.html
  • https://discourse.wicg.io/t/onscreen-keyboard-api-to-obtain-dimensions-presence-of-the-keyboard/674
  • https://discourse.wicg.io/t/exposing-a-users-input-modality/1067
• https://github.com/w3c/csswg-drafts/issues/1911

[birtles]

For this use case I think all we're interested in is whether or not the user is likely able to use a keyboard for shortcut keys. There may be some keyboard layouts that don't support certain shortcut keys (e.g. keyboards without a function key row) but handling that seems like a separate problem (as does tailoring the shortcut key descriptions to match platform conventions).

Even if the shortcut key is just a you're not likely to use it as a shortcut key on a virtual keyboard, but you are on a physical keyboard.

[samhed]

Another use case that is relevant is simply if there is a physical keyboard available or not. On devices where no physical keyboard is available, our web app wants to show a button that opens a virtual keyboard (by moving focus to a textbox).

As mentioned in the initial post, until we have something like this, we're forced to make the bad assumption that touch devices have no physical keyboards (we use any-pointer: coarse).

[emilio]

Strawman proposal, wdyt of: @media (keyboard: physical|virtual|none)?

[emilio]

If we need further distinctions we can add them in the future, but that should allow most use-cases without raising a lot of hard-to-answer questions?

[tabatkins]

Yup, I think we can extend it later to allow, say, (keyboard: physical numpad) if that's a distinction we want to expose.

When would you have keyboard: none?

[emilio]

Printing perhaps? Other non-interactive media?

[tabatkins]

Yup, printing definitely makes sense, just wanted to check.

[birtles]

Some extra background regarding the motivation and timing for this, Chrome is switching to desktop mode for premium tablets which is going to cause trouble for sites using the UA string to determine if the device has a keyboard or not.

From the article:

it is a good time to confirm that all interactions with the desktop versions of your site work well for users with and without an attached physical keyboard

In my experience, you currently need to use the UA string to avoid false negatives on some devices.

[Crissov]

Media Queries are not intended to be used with CSS only. If they were limited to styling, I’d agree that there should only be minimal support for keyboard detection. If the displaying keyboard (shortcut) hints use case is valid, MQ would need to be powerful enough to distinguish different types of keyboards and layouts.

For the presence of a keyboard, MQ should distinguish between always and on demand, but perhaps more importantly between separate from the viewport and semi-transparently overlaying, opaquely obstructing and intrusively shrinking it. For virtual keyboards in particular, there may be reasonable desires to know whether it is used by swiping or typing/pressing and whether 1 finger or thumb, 2 fingers/thumbs from separate hands or all fingers 1 from 1 or 2 hands are typically used. (Key chaining is a bit exotic in alphabetic cultures, but may abhustend describe Asian IMEs in particular.)

For some keys, it could be enough to provide access to locale or system-specific labels elsewhere, e.g. Cmd (Mac) vs. Ctrl (EN) vs. Strg (DE). That is probably outside the scope of both, CSS and MQ (except in env() perhaps).

The exact availability of key group levels like shift [lock] and alt/option, perhaps does not need to be exposed to authors.

Authors would probably want to know sometimes whether the keyboard provides 1D sequential navigation of interaction focus (e.g. with tab) or 2D spatial viewpoint navigation (with 4 to 8 arrow keys, d-pad or joystick, but also mouse wheel and page +/-, home and end keys).

I believe there could be valid use case for information about the following key types as well:

• compose, dead (accent) or hold (variants)
• escape/cancel/back or enter/return/submit/confirm/select/OK
• forward delete/erase and backwards delete/backspace
• (context) menu
• digits (numeric input)
  • row
  • phone-pad, T9
  • calculator-pad
• letters, syllabograms (linguistic input)
  • words (suggestions, autocomplete)
• colors (teletext)
• playback (like play/pause, skip, stop, record)

[smfr]

Some fingerprinting thoughts: I don't really want pages to be able passively to distinguish tablets with a hardware keyboard attached vs one without. I also don't want pages being able to tell if I have a keyboard with a number pad vs one without.

[css-meeting-bot]

The CSS Working Group just discussed [mediaqueries-5] Detect physical keyboard (keyboard capable of shortcuts?).

The full IRC log of that discussion

<Frances> Florian: Seems useful to have media query to see if you have a keyboard or not for keyboard shortcuts or a layout with keyboard shortcuts. What is a keyboard and what keyboards do we want to detect?
<bkardell_> q
<Frances> Florian: onscreen keyboards, virtual keyboards, need to define with use cases on a variety of devices before answering the questions such as privacy concerns.
<astearns> q+
<Rossen_> ack astearns
<Frances> Brian: fingerprinting could be wary of it
<flackr> q+
<Frances> Alan: Displaying shortcuts only if there is a keyboard to accomplish shortcuts, could possibly be bad if we don't notify the user
<Rossen_> ack flackr
<Frances> flackr: Developers can already infer whether there is a keyboard or not, exposed through the visual api only after interaction.
<vmpstr> q+
<astearns> s/notify the user/notify the user, because they might decide to plug a keyboard in to use the shortcuts/
<Rossen_> ack vmpstr
<flackr> s/visual api/visual viewport api
<bkardell_> s/fingerprinting could be wary of it/I was going to agree with smfr on the thread that fingerprinting/privacy were something to be aware of and maybe make us wary of this
<Frances> vmpstr: There could be use cases for games showing on screen controls vs not when there is a keyboard attached
<Frances> Florian: These are the type of questions we need to answer.

[xiaochengh]

I think this issue is highly related to the Keyboard API and should be discussed together.

To prevent privacy issues, the Keyboard API requires a permission. For the same reason, a CSS feature that works with the physical keyboard must also require a permission. Does CSS currently work with permissions at all?

I'm also wondering if we really need a new CSS feature:

• Right now authors can already detect the physical keyboard (+layout) and apply styles conditionally
• Since authors may want various detailed info than just the existence of a physical keyboard, a CSS feature might end up with very complicated syntax, which looks bad to me; while JS seems more appropriate for this task

And even if we do need a new CSS feature, I think we should wait until the JS API reaches a more mature status.

[goury212]

So why not target desktops only? It's fair to assume that only desktop / laptop users have physical keyboards

If you want to test the keyboard keys, if your keyboard is not working well, if there are faulty keyboards, or if you want to try your computer keyboard, then this free online tool is for you.