Open zouhir opened 4 years ago
I have 2 question:
env(fold-width)
and env(fold-height)
? also isn't 100vh
=== env(fold-height)
?top:0; right:0
=== top: env(fold-top); right: env(fold-left)
I'm a little concerned about extending this out in the future, and I'm wondering if maybe we should be splitting this out a bit more e.g.
display-span-x: <integer[1,inf]>
display-span-y: <integer[1,inf]>
We might not be able to return all the fold information for 5 folds via an environment variable right now, but at least the media query can be a little more accurate and can extend more gracefully into the future possibilities of N panels.
The CSS Working Group just discussed Proposing new CSS primitives to enable great web experiences on foldable & dual-screen devices
.
I'm wondering if flex and grid need some kind of extension to allow positioning of "hinges" in them , in a way that the flex/grid layout algorithm could take into account (possibly but no necessarily including a gap at that point). In the case of a flex, that would probably behave sort of like a flex-wrap in terms of laying out and sizing items within each piece of the flex, but unlike flex-wrap, the two pieces wouldn't be on different lines, but contiguous to eachother. If we had something like that, it feels that the environment variables being discussed here would be a lot more useful.
Even then, another difficulty with the environment variables is that they give you the position (and size) of the hinge relative to the screen/viewport, but to be able to use them in css for the layout/position/sizing of things that aren't the root element (or absolutely positioned with the root as a containing block), that's not super useful: you'd need to know about whether your parent element / parent formatting context is being cut by a hinge, and if so where in terms that are relative to your layout space.
Without that, it feels like you're going to have to use complicated calc expressions to duplicate the way your ancestor chain has been sized / position in order to figure out where the hinge falls with regards to you.
Multiple screens is going to lead to some fun in general with media queries. Nearly all queryable properties can differ depending on the screen being used. It would be... bad... not to allow content to query the capabilities of each screen it is using.
It is hardly unlikely that someone is using a laptop (close to user, high dpi) connected to a beamer (far from user, different aspect ratio). Or just a big TV sized screen.
And then there is the issue of hardware acceleration. Screens aren't equal, but neither are graphics cards. Ain't no rule that all graphics cards in a machine are equally capable. Heck, I bet that there are setups that have multiple candidate graphics cards that share the same screen with a software controlled switch. In fact, I hear that's a common setup on laptops.
Is there any specific reason why the spanning and environmental variables are targeted towards dual screen devices? My main concern is that there should be a way of identifying a foldable independent of if it is seamless or with a seam. This proposal can be useful even on seamless devices (I’m thinking here of Galaxy Fold and other upcoming devices) to adapt the UX based on the state of the device. At the moment we are working in Samsung on an explainer that will shed more light on scenarios and new capabilities and how they would combine with the spanning media feature and fold environmental variables.
Out of curiosity, in a device like the Galaxy Fold or the Motorola Razr, and according to the definition provided in the explainer about ‘spanning’, would it resolve to none? If that’s the case, it might be better to specify the ‘single-fold-vertical’ or ‘single-fold-horizontal’ and appropriate env fold variables to potentially accommodate the content on the web page?
To summarize, my concern is with the ‘multiple screen’ wording that keeps appearing in the explainers, since these concepts are useful for single screen devices as well.
@fantasai- That is a great suggestion - your proposed features would indeed make extensibility much easier for authors to reason about, and removes the hurdle of future form factors needing to come up with various individual values to describe available configurations. We'll look into testing the concept and validating with authors that have been keen on trying out these features.
@frivoal- Thanks for the feedback and use case! We had not fully considered the best way to treat containers with children that want to avoid the fold. Many of the design patterns suggested for the Microsoft form factors have focused on, as you mention, a high level splitting of content, or absolutely positioning things around the hinge/fold. We'll take another look and see if the current environment variables can be harnessed or if we need some additional mechanisms to enable these scenarios.
@diekus- The proposal has is not aimed at making the spanning and environment variables exclusive to dual screen devices. I'll go back and re-review the wording to clarify this point.
We're trying to expose the underlying system's description of the semantics of the window. For devices like Galaxy Fold or Motorola Razr, if the window manager reports a fold (I believe the currently discussed abstraction for Android is DisplayFeature), then spanning
would resolve accordingly. Additionally the information the system exposes regarding the geometry of the fold would populate the environment variables.
If the window manager doesn't expose this information (i.e. expects apps to just use the available space and not treat it as logically split into content regions), then spanning
would resolve to none.
The media query would need to be able to handle multi screen setups. Users could have a huge variety of set ups though.
There are sensible 3 screen set ups like this:
All the way up to something ridiculous like this:
The users display also might not even be rectangular:
The 3 screen set up is sensible to support. Do we want to support the more intense screen set ups as well though?
fold-top
,fold-left
,fold-width
andfold-height
.
It would feel very unCSSy to not have respective bottom and right fold positions as well.
Overall, this issue raises some important points on design constraints, but the actual representation thereof in CSS, MQ or environment variables requires some additional thinking.
Some display mosaics simply represent a fixed segmentation of a common shared planar surface (i. e. the viewport), like an infinite virtual wall. The static edge width around each screen (and thus between any two neighboring screens) may vary, often approaching zero for a seamless appearance, and are important to consider in layout because the wall may either be split at these frames or be covered virtually by the physical bezels, i. e. continue invisibly beneath them. In some cases, like a detachable mobile second screen that extends the desktop plane, the distance and relative (planar and even spatial) location is even variable. In other cases, the wall (or its displayed projection, i. e. the viewport) slants by a fixed-angle bracket or by a variable-angle hinge at the joint, where the latter becomes a fold if only 0° and 180° are stable positions. When this knee is smoothed out and hence seamless, it is a bend, otherwise a sharp kink. (All terms up to bikeshedding by native speakers of course.)
For the simplicity of a first level, we may assume that the gutter between all segments is the same and distributed evenly, except we would distinguish horizontal from vertical ones.
We should also note that the classic spread of left and right pages in a printed book or magazine is conceptionally very similar to two adjacent screens. They can also be more complex, e. g. in centerfolds. If media can be folded completely at a joint, the distinction between recto and verso or front and back also becomes important.
segmentation-x
or horizontal-segments
: maximum number of horizontal viewport segments spanned segmentation-y
or vertical-segments
: maximum number of vertical viewport segments spanned segmentation
: none | [vertical | horizontal | both] [continuous | separate]?
segmentation-gutter[-x|-y]?
: [seam | frame | bezel]#
or [invisible | visible | covered]#
segmentation-joint[-x|-y]?
: [flat | bend | bracket | hinge | fold]#
or [straight | [curved | sharp] [static | variable | alternate]?]#
env(vertical-seam-width)
env(horizontal-seam-width)
env(vertical-seam-angle)
env(horizontal-seam-angle)
Desktop setups with multiple screens would often have angles between 0° and 45° with A cubic cave, with (up to four) walls, optional floor and ceiling, would usually have zero-length seam widths and right seam angles.
For curved surfaces (and spheric caves, i. e. physical rooms, or virtual rooms around the user), we might want something similar:
env(bend-radius)
env(horizontal-bend-radius)
env(vertical-bend-radius)
Out of curiosity, in a device like the Galaxy Fold or the Motorola Razr, and according to the definition provided in the explainer about ‘spanning’, would it resolve to none? If that’s the case, it might be better to specify the ‘single-fold-vertical’ or ‘single-fold-horizontal’ and appropriate env fold variables to potentially accommodate the content on the web page?
Something to add to this is that in single screen devices like the Galaxy Z Flip or what Intel showed as the Horseshoe bend concept, the device is partially folded and the fold may or may not be usable (touch not working reliably or image distorted).
So indeed the single-fold-vertical/horizontal properties could be defined even if there is no physical seam because the platform could or should define one arbitrarily. Also the platform/vendor may choose to give the opportunity for the developer (if she/he wants) to provide a split layout (top and bottom of the app being different) because it makes more sense for the website.
To summarize, my concern is with the ‘multiple screen’ wording that keeps appearing in the explainers, since these concepts are useful for single screen devices as well.
I second this as per above.
Thanks @fantasai and @Crissov for your feedback, since this thread have quite a few feedback items, I thought it might be worth breaking them down into a separate forks \ issues.
I have opened this issue to change our currently proposed spanning: <string>
to something along the lines of what you have posted above display-span-x: <integer>, etc
you can find it here: https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/235.
I have also went ahead and requested feedback from early API customers and I'd expect them to chime on that issue shortly.
@Crissov , I have opened an issue to addd env(fold-right)
and env(fold-bottom)
-- as I described there, I believe it'll be more CSSy and will provide better developer ergonomics.
https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/236
The problem with the term "fold" is that it is too device specific.
Would you call the gap between 2 monitors a "fold"?
Maybe, but it isn't as obvious in that context.
Great point @Dan503 -- I was actually meaning to reply to your comment regarding multi-monitor.
I completely understand that the concepts are related (dual-screen, foldables and multi-monitor) and I am not opposed at all to consider multi-monitor setups, therefore, if we were heading down that path, I agree we need to find a better name than fold-*
.
While working with developers who are planning to add support for dual-screen & foldable devices, multi-monitor never came up as something they are wanting to target or apply their dual-screen layouts to. So, if we can have a customer with a good use-case for multi-mon, that'd be ideal, until then, I think we can brainstorm what we need to do to enable that path, some questions I have:
the right side in the figure above is not good for any content, and media queries does not yet enable us to check how big that side is, example:
@media (display-span-x: 2) and (min-width: 900px) and (calc(100vw - env(fold-right) >= 400px) {
/* only now apply spanning layout, because it's more than a tablet with enough room on the right side of the fold */
}
I am worried developers will start using device specific ratio
or other stuff to only apply spanning for portable foldables if we don't offer multi-monitor exclusion.
And I would add that the fact that often multi monitors have heterogeneous setup where resolution/dpi and zoom level may differ between the monitors (different brands/model or spanning from laptop screen to monitor) therefore it makes even harder to span across screens in a reliable fashion. Window managers struggle today with that, just drag a window from one screen to another and see how it gets redrawn and layout again if the monitor configuration differs. When the window spans across two "different" screens they also look weird and "zoomed in" (see my picture) : on my main screen it doesn't look right for e.g. (and it seems that they're are some decision in the WM on when to pick a given state, often based on how much the window is spanning on a given screen). So I don't know how we would expose that to the content.
In the illustration of @zouhir it will be hard to align the right yellow piece (even if it was full screen) to the left yellow piece, so it may make the content look weird unless the author purposely create a split UX (not to mention the physical aspect of it).
I think at the end we're opening a can of worms by enabling this on multi monitor setup. And the fact that multi monitors have been around for decades and nobody asked for features like this (aka spanning on multi monitors) is also a strong indicator that maybe we don't need that.
I would like to point out this btw which somewhat try to address multi monitor and window placement : https://github.com/webscreens/window-placement/blob/master/EXPLAINER.md
Those are some good points @darktears.
A good middle ground might be to only work if the screens are the same resolution. But then a foldable device comes out where one screen is slightly smaller than the other screen and now these foldable things in our CSS don't work anymore for this new device.
I want to also bring up that we seem to be focusing a lot on media queries but what would be more useful I think is a way to use the size of the current display when setting the size of elements.
I'm thinking that we would need new dw
,dh
,dmin
,dmax
units. Basically a copy of the viewport units (vw, vh etc.) but they are percentages of the current display rather than the full width/height of the browser window.
.fullScreen {
width: 100dw;
height: 100dh;
}
We might also be able to use this as an opportunity to enforce a fix for the problems with the existing viewport units that drive web developers insane. The problems being that 100vh
doesn't take the size of the Chrome on mobile into consideration and 100vw
doesn't take the width of the scrollbar into consideration.
100vh
means that when the page first loads on mobile, content at the bottom of the screen isn't visible until the user scrolls. If the content is not taller than the height of the screen, this forced scroll the user has to do is a terrible UX.
100vw
ends up introducing a horizontal scrollbar whenever it is used on a desktop browser with more content than the screen can fit.
We'd like to discuss options we've been exploring to update the syntax to be more forward-compatible at the upcoming vF2F.
The CSS Working Group just discussed Proposing new CSS primitives to enable great web experiences on foldable & dual-screen devices
.
This stuff does not seem at all easy to use. I can't imagine that adoption by web developers would ever be all that high.
Is there any plan for something simple, like a css function that adjusts a size or coordinate to snap to a natural boundary position? Something like:
nav {
--smallest-acceptable-width: 300px;
--ideal-width: 40vw;
--largest-acceptable-width: 60vw;
width: natural(var(--smallest-acceptable-width), var(--ideal-width), var(--largest-acceptable-width));
}
This stuff does not seem at all easy to use. I can't imagine that adoption by web developers would ever be all that high.
It's a media query opt-in, I don't find particularly complicated to use. See examples here: https://github.com/foldable-devices/demos
Is there any plan for something simple, like a css function that adjusts a size or coordinate to snap to a natural boundary position? Something like:
nav { --smallest-acceptable-width: 300px; --ideal-width: 40vw; --largest-acceptable-width: 60vw; width: natural(var(--smallest-acceptable-width), var(--ideal-width), var(--largest-acceptable-width)); }
How you would handle the physical hinge covering content with that proposal? Typically, when designing for dual screen or foldable in split mode, the idea when spanning is not to have your content be the width of the 2 (or n) screens because the hinge may cover content or the fold could create an interaction issue (think if the device is half folded). The screen estate being bigger and the expected interaction being different typically would require the developers to rethink the layout slightly. Matching the MQs make the developer think that there are now 2 segments of content that can be displayed and thus enable a new layout (often a two pane or reflow the current one to take the hinge into account). This is an opt-in which is good.
Microsoft put up some nice videos that showcase some new layout patterns. See https://www.youtube.com/user/surface/videos
I encourage anyone interested to join this TPAC breakout session to discuss alignment opportunities between JS API proposals for foldables and multi-screen environments. That will continue discussion from the Second Screen CG meeting; minutes. Thanks!
Is there any plan for something simple, like a css function that adjusts a size or coordinate to snap to a natural boundary position? Something like:
nav { --smallest-acceptable-width: 300px; --ideal-width: 40vw; --largest-acceptable-width: 60vw; width: natural(var(--smallest-acceptable-width), var(--ideal-width), var(--largest-acceptable-width)); }
How you would handle the physical hinge covering content with that proposal? Typically, when designing for dual screen or foldable in split mode, the idea when spanning is not to have your content be the width of the 2 (or n) screens because the hinge may cover content or the fold could create an interaction issue (think if the device is half folded). The screen estate being bigger and the expected interaction being different typically would require the developers to rethink the layout slightly. Matching the MQs make the developer think that there are now 2 segments of content that can be displayed and thus enable a new layout (often a two pane or reflow the current one to take the hinge into account). This is an opt-in which is good.
If the nav is aligned left, and the left edge of the nav is positioned such that the distance from that edge to the left edge of the hinge falls within the range the author has contemplated, that the width of the nav will end up being exactly the distance to the left edge of the hinge. Same with align right and right edges. Center alignment is naturally more perilous. In all cases without a good solution, the nav will end up with the author's ideal width, which may not be ideal for the user of the device, but then there is no fully generic solution to creating a "great web experience" on all possible foldable displays. My syntax makes it quick and easy to support foldable devices that the web site's existing design intuitively should support.
This snap-to-natural-boundary behavior should be supported on at least left, right, width, top, bottom, height, margins, and paddings.
My colleague @dlibby- and I have published an explainer on MicrosoftEdge's Explainers repo where we have proposed a new JavaScript API and a CSS media query with a set of pre-defined env() variables. Both aim to enable developers to lay content out effectively in a window that spans multiple displays, and more specifically for dual screen devices.
We wanted to communicate our thought process behind why are we proposing the new CSS primitives and collaborate with the working group to help web developers continue to build great responsive experiences on the web for every device that runs a browser.
The flexible form factor trend
The trend started in 2019 when Samsung, Huawei, Asus and others released their foldable & dual screen devices, and in 2020 Microsoft will be adding 2 devices, Surface Neo and Surface Duo to this flexible device category.
Why are we proposing new CSS primitives?
When the browser window is spanning across the device fold, informing developers about the fold orientation and display boundaries can help them design great experiences. Here’s a quick overview of possible patterns and opportunities for the web platform when CSS can provide the necessary information on this class of devices:
1. Bringing wider-screens UI patterns to smaller portable devices & Creating enlightened UIs on the web
Due to screen size limitation, traditional portable touch-screen devices rely a lot on “stacking views” where clicking on an email in your inbox list for example will cause that whole inbox list view to be replaced with the selected email content view. This behavior usually creates additional navigation steps while wider screen devices have a more natural configuration where the email inbox list view and email content view are side-by-side.
When the browser is spanning across the 2 screens on a dual-screen device developers should be able take advantage of the wider-viewing area and also snap to the natural boundary of each screen.
We also strongly believe that re-creating patterns that make sense is not the only PRO here - this device category with 2 displays and flexible hinge can offer unique opportunities for developers and designers to create new experiences as long as the web platform provides necessary information (eg. screen boundaries, fold orientation).
2. Easily improve existing sites & UI components
Developers might not want to introduce major UI changes for this class of devices and would just want to simply move some components around. In the example below, it makes more sense for the modal dialogue to avoid the device fold, whether the foldable device is seamless or has a seam; and we want CSS to offer the capability to match this device class and allow developers to progressively enhance their site.
Operating systems & Native app platforms status
Microsoft has released some information about OS & native app platform APIs and SDKs.
Android SDK & Emulator for surface Duo
Windows 10x API documentation
Microsoft blog post about app patterns for dual-screen devices
We’d like to have a standard declarative web API browser vendors can implement so website and PWAs can match native app platforms capabilities.
Proposed CSS primitives and general design principles
The 'spanning' CSS media feature
A simple CSS media feature to test whether the browser window is spanning across multiple displays on a dual-screen device.
The
spanning
media feature value can be one of the following keywords:single-fold-vertical
: This value matches when the layout viewport is spanning a single fold (two screens) and the fold posture is vertical.single-fold-horizontal
: This value matches when the layout viewport is spanning a single fold (two screens) and the fold posture is horizontal.none
: This value describes the state of when the browser window is not in spanning mode.Device fold CSS environment
env()
variablesWe propose the addition of 4 pre-defined CSS environment variables
fold-top
,fold-left
,fold-width
andfold-height
. They represent the fold's (device hinge) rects and web developers can utilize those variables to calculate each screen segment size at both landscape and portrait orientations.Some notes on our general design principles
Our first attempt tried to find a way to infer the hinge orientation using the existing orientation media feature. However, we quickly found that this is something we can’t generalize; a device can be in landscape orientation but has a vertical fold (folds like a book) another device in landscape may have horizontal fold (like a laptop).
It’s much simpler to design a JavaScript API that returns an array for DOMRects and have it scale for a device with
n
number of screen; the reason is in JavaScript mappings are more direct, we have concept of for loops, conditionals and arithmetic operations. These can be combined to enable the developer to react to arbitrary topologies. For CSS, we decided to scope our solution to this new class of device that has specific design principles around how the displays are utilized by an application. However, we believe the new media feature can be extended with new values that match future form factors down the line.Final thoughts
We'd like to hear your thoughts and feedback, so please feel free to use this GitHub thread as a discussion forum or open issues for us on our Edge Explainers GitHub repo.
Additional material you might be interested in looking at:
Demo URL
Demo source code
Colored boxes positioned absolutely example