Picker-style API

[hober]

Hi,

I got here from w3ctag/design-reviews#413; @plinss and I took a look at this during the TAG’s Cupertino F2F.

We wonder if a picker-style API could be simpler, less powerful, and still fulfill the use cases. It would need:

• A predicate and/or media query to answer the question “are there more than one screen? (AKA should we bother to show a button to invoke the picker?)”
• A method for invoking the picker, perhaps with a bag of constraints (“I prefer portrait screens that are at least 300px tall”, that sort of thing)
• A way to target a screen when opening a window
• Possibly also a way to move an existing window to the screen vended by the picker API.

[michaelwasserman]

Thanks for taking some time to consider this proposal! I'd like to better understand the described picker-style API. Would that roughly be a JS API requesting that the browser show a dialog or interstitial for the user to select a screen that would host a given (new or existing) content window?

I'd also like to expand on a few use cases where a more general API, like those described in the Screen Enumeration and Window Placement proposals would be desirable, and a picker-style API (IIUC) might not be optimal or sufficient:

1. A financial dashboard opening multiple windows across multiple screens, using the overall screen-space information to produce a grid-like placement of windows. Showing a picker for each window could be cumbersome and may not suffice to produce the overall desired window placement. Further, the proposed API would allow the dashboard application to persist and restore the layout between user sessions.
2. A slide deck editor opens a presentation on an external display and speaker notes/controls on an internal display. The target screen for each window is straightforward to determine given multi-screen information, without needing to prompt the user for each window. Additionally, a mechanism is needed to request the fullscreen slideshow on the target display. A similar scenario arises when a meeting room system has a touchscreen controller and a large video display with readily apparent target screens for each window.
3. A single window spanning multiple screens tailors its content to appear optimally in that configuration (eg. a list of items on one display, and a view of the selected item on the other display). Without information about the multi-screen space, it's unclear if the content could achieve this enlightened layout.

We definitely need to continue balancing API utility with simplicity, and to limit the potential for abuse while still providing something powerful enough for the most requested and user-beneficial use cases. I look forward to hearing more of your thoughts here; thanks!

[Garbee]

If I'm understanding correctly by "Picker-style API" it's being asked that an API is provided instead that developers can request with constraints what they need, a user then selects (if any are available) to give access to the program (much like the contact picker, but capable of having restrictions put into the generated list.)

This to me adds a high amount of friction to apps that take advantage of this API. Along with an added intricacy that if the monitor space changes, does the user then need to re-approve even existing selections since now there could be more machine identifiable information that could be provide?

We need to find a balance between having the data protected from just any site being able to get all of it and letting apps that take advantage of it not end up with a cumbersome (at best) user experience. Picker-gated access feels a bit too unbalanced.

I think it could work together with the option of still receiving perpetual access. Perhaps if you're not an installed PWA, then you can't request outright full access. But if you're installed then that capability opens up for request so you can always handle whatever you need to do.

[michaelwasserman]

Thank you again, @hober for the broad perspective, high-level analysis, and breakdown of approaches here and in #152.

Do you have any thoughts regarding my earlier response?

I like the prospect of allowing requests for limited screen information, such as a single multi-screen bit, a count of screens, access to a screen matching/maximizing certain parameters, etc. That said, I believe that some stated use cases would be cumbersome for users and developers, if not impossible, if a native picker or more strictly limited APIs were the only tools available.

[michaelwasserman]

The updated and consolidated proposal explores use cases that could not be solved by a picker-style API. The additional explorations document covers the topic of User agent screen pickers and limited power APIs. If there are specific aspects of this topic that deserve further exploration or discussion, I welcome that feedback through new issues on the webscreens/window-placement repo, or feel free to reopen this issue if that seems more appropriate; thanks!