[Virtual Keyboard API] A declarative mechanism to enable overlaycontent?

travisleithead commented 3 years ago

Porting over from MSEdgeExplainers repo https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/416

@snianu filed:

Since we already have CSS env variables for keyboard insets, should we also have a declarative mechanism to enable overlaycontent? Something like a meta tag?

bramus commented 2 years ago

As suggested in #13 (duplicate):

An extension to meta name="viewport" (or perhaps a new CSS at-rule or property on <html>) to control the behavior seems like something handier.

Something like this:

<meta name="viewport" content="width=device-width, initial-scale=1.0, virtual-keyboard=overlays-content">

Or this:

:root {
  virtual-keyboard: overlays-content;
}

(names to be bikeshed but you get the point)

andruud commented 2 years ago

Meta viewport would work, but the CSS approach could create contradictions or loops, e.g. say you do this while the keyboard is open in a non-overlay-by-default browser.

@media (height < 900px) { /* True, because the open keyboard makes the viewport/ICB small */
  :root {
    virtual-keyboard: overlays-content; /* Applying this resizes the viewport/ICB, making the MQ 'false' */
  }
}

snianu commented 2 years ago

@BoCupp-Microsoft FYI..

andruud commented 2 years ago

virtual-keyboard=overlays-content

BTW, what would we call the opposite value? (Considering that overlays-content may be the default).

flackr commented 2 years ago

virtual-keyboard=overlays-content

BTW, what would we call the opposite value? (Considering that overlays-content may be the default).

So I was just testing this and it seems that this overlaysContent API is very different from the way that the virtual keyboard overlays content on chromeos or iOS.

The demo is https://output.jsbin.com/cosopok/ On ChromeOS and iOS, the layout viewport does not change when the keyboard shows, and you can scroll to the bottom of the page with the keyboard open (seeing both the input element at the bottom and the bottom of the measuring bar on the left).

However, when overlaysContent is enabled, you can never scroll to the bottom of the page, meaning you can't see what you're typing into the bottom input.

So unless we change the overlaysContent behavior, this is a different orthogonal feature and will likely never be the default.

snianu commented 2 years ago

@flackr overlaysContent when set to true, prevents resizing of both layout and visual viewports when the virtual keyboard shows up. Resizing the visual viewport (without resizing the layout viewport) gives the user the ability to pan around the layout viewport within the smaller visual viewport. If we aren't resizing the visual viewport (so no resize event) or the layout viewport, then you do need some event and/or CSS env variable to allow the author to relayout the page. That's why the VK API has geometrychange and env(keyboard-inset-*) so the web authors can adjust just the input control where the user wants to type that gets hidden behind the keyboard without affecting the layout of the entire page. On iOS, the visual viewport is resized, but not the layout viewport. This comment has an example and discussions between us and Apple devs which might be relevant to your question.

bokand commented 2 years ago

However, when overlaysContent is enabled, you can never scroll to the bottom of the page, meaning you can't see what you're typing into the bottom input.

From the explainer, I think this is intended - it's meant for cases where the app will reposition elements itself and IIRC was to prevent resizing the visual viewport on dual-screen devices where the keyboard only occupies half the (dual-screen) viewport.

It'd be nice if we could choose behavior, i.e.

virutal-keyboard: resize-layout        // Changes ICB size, causes reflow

virtual-keyboard: resize-visual        // Changes visual viewport size, no reflow but obscured
                                       // content can be scrolled into view

virtual-keyboard: overlays-content     // Doesn't resize anything, content must adjust itself

bramus commented 2 years ago

To add/clarify:

virtual-keyboard: resize-layout = Current Android behavior
virtual-keyboard: resize-visual = iOS/CrOS behavior

flackr commented 2 years ago

I think it may be simpler if we change android to match iOS / CrOS, and then if developers want resize-layout behavior they can effectively get it with overlays-content + a simple style which excludes the keyboard area such as the grid example in the spec.

bokand commented 2 years ago

Ah, yup, an app can fairly easily simulate resize-layout so I agree changing default to match iOS/CrOS and overlays-content covers all the cases

css-meeting-bot commented 2 years ago

The Web Editing Working Group just discussed declarative mechanism to enable overlay content?.

The full IRC log of that discussion

<Travis> topic: declarative mechanism to enable overlay content?
<Travis> github: https://github.com/w3c/virtual-keyboard/issues/3
<Travis> snianu: mutliple issues here
<Travis> .. overlayContent was set on VK (in JS) via the navigator object...
<Travis> .. from TAG review, they requested an env variable for inset (for geometrychanged event), then these CSS variables can be used to understand occluded content.
<Travis> .. there was some addition concern--if we have CSS env variables, then why set overlayscontent in JS?
<Travis> .. perhaps there should just be some HTML declarative thing?
<Travis> .. (that's the recap)
<Travis> .. was a suggestion to never resize the visual viewport (on overlays content set)
<Travis> .. (today on Windows we change both layout & visual viewport)
<Travis> .. then the overlays content would only control the layout viewport
<Travis> .. (looking up some examples to share to help explain the details)
<Travis> https://github.com/w3ctag/design-reviews/issues/498
<Travis> snianu: I think we can discuss this async...
<Travis> johanneswilm: Are you seeking input on seeing if making the suggested (or some other proposal)?
<Travis> .. or are you asking for agreement on the suggested syntax?
<Travis> .. not sure if Safari is planning to implemented overlayscontent, and if so, what the behavior should be?
<Travis> .. should it adjust layout + visual viewport.
<Travis> .. currently only chromium supports.
<Travis> travis: whenson, do you have any feedback on the declarative mechanism?
<Travis> whsieh_: The behavior would apply to any element that is a focusable target for the keyboard to appear?
<Travis> .. should the behavior "descend" into other elements? e.g., with iframe?
<Travis> snianu: currently we're excluding iframes.
<Travis> whsieh_: My intuition (for like messanger) might be useful in the main chat box... but in search field is may not be as appropriate (or in some other view).
<Travis> snianu: ex: on dual-screen examples, you can split screen into two. On one screen you have a search element. Current behavior: keyboard covers both screens.
<Travis> .. with overlayscontent flag it doesn't affect the layout of the whole page, just the element in question.
<Travis> .. that was the original scenario that inspired the flag.
<Travis> .. ex: in messager, w/ VK, you might want to adjust just the textbox and not the entire layout.
<Travis> whsieh_: for a meta tag, does the web engine need to respect dynamic changes?
<Travis> .. for declarative mechanism they may need an attribute on an element.
<Travis> .. whsieh_ the flag on the navigator... object already offers this flexibility.
<Travis> .. stepping back, I'm curious what the declarative mechanism (on the meta tag) would offer?
<Travis> .. if we did add a declarative, I would be more in support of an attribute (i.e., like inputmode or similar)
<Travis> Travis: I don't like meta tag for cases where the state will dynanically change (meta is usually for set-and-forget states), which this doesn't seem like it's really static...?

w3c / virtual-keyboard

[Virtual Keyboard API] A declarative mechanism to enable overlaycontent? #3