Frameless media feature and CSS window controls delimiters for full customization

[ghost]

Issue:

It appears that the proposed solution does not give to developers full control over the title bar, but only opens a part of it to customization.

It means that there is no way for the developers to customize and style the window controls themselves. Moreover, the proposed solution is only available for PWAs but could benefit traditional websites as well.

Solution:

A solution to that issue could be to have a frameless display mode as a CSS media feature, which would display the current website in a totally frameless window.

Then, specific CSS window control delimiters such as specific CSS class names could be used internally as an actual overlay to native functions for closing, minimizing and maximizing the window, giving to developers the ability to customize the controls.

Entry into the frameless display mode could be set via user action through menu option and shortcut for traditional websites, or programmatically, through the PWA manifest and on behalf of the user, during the PWA installation and launch.

Working example:

A working example of that solution (customization part) can be found in NW.js applications (a solution based on chromium similar to electron for creating desktop applications), in the following repository: nwutils/frameless-example

Examples of CSS class definition for window control, and their application to the HTML markup show that the developers have full control over the look and feel of the window, with the ability to use HTML and CSS to style the window controls.

[Malvoz]

Keep in mind that one of the Goals is to ensure accessible controls:

• Ensure accessible user control of the app window is maintained (at minimum - UA supplied minimize, close and drag window controls)

[Malvoz]

A working example of that solution (customization part) can be found in NW.js applications (a solution based on chromium similar to electron for creating desktop applications), in the following repository: nwutils/frameless-example

Those controls are not accessible via keyboard. Seemingly, a keyboard-only user would be trapped in the window.

Edit: Though it's possible to alt + tab out of the window, not all users can perform such key combinations. Also I'm not sure there's a way to escape the window using TalkBack, for example.

[ghost]

@Malvoz

Keep in mind that one of the Goals is to ensure accessible controls

Ensuring accessible controls should not be a difficulty. If specific CSS class names are effectively bound to native window controls, ensuring accessible controls would just be a matter of checking if the webpage stylesheets actually contain those rules, during the CSS parsing phase.

In the case of a transition from a framed window to a frameless window via user action, if the stylesheets do not contain those rules, then the entry into frameless display mode would be cancelled.

For PWAs, the presence of those classes in the stylesheets could be enforced by design (when the manifest contains the frameless display-mode feature). If the CSS rules are not present, or even if the controls are hidden to the user at launch time in CSS, the browser could also potentially automatically revert back to a framed window.

Those controls are not accessible via keyboard. Seemingly, a keyboard-only user would be trapped in the window.

The example above is only to showcase the ability to style the window controls with HTML and specific CSS delimiters. Keyboard accessibility can actually be defined via javascript. However, developers can actually override the default keyboard shortcuts of a normal browser windows, so it does not create a new problem on this side, as a website can already traps a keyboard-only user into the window. If the browser actually ensures that visible controls are present during the CSS parsing, then frameless mode would be in parity with the default display mode as far as keyboard accessibility is concerned.

[ghost]

Those controls are not accessible via keyboard.

I now see what you pointed out. From the operating system perspective, it is true that the DOM is not visible, so those window controls would not be accessible from outside of the browser.

But as we are dealing here with web technologies, we should not expect 100% full feature parity with native desktop applications, such as the ability to control the window from outside of the browser.

Once the user is engaged into the frameless window, keyboard control is just given to the application, like in the normal default browser window, without expectation that a third-party software could programmatically control the window controls.

Edit: if a developer wants to be able to control the window from outside of the browser/application, then a webview runtime is a better option.

[amandabaker]

Hey @abflow,

Thanks for the feedback! Although it would be great for developers to have that level of control of the window, it's out of scope of this feature. Feedback from the TAG review and other security reviews has brought up concerns of using this feature to spoof the browser.

Since this feature enables PWA's to paint in areas where the window title bar would have been, we identified a concern of impersonating a browser. Most modern browsers today have their tabs UI in the window title bar area, something that's hard to spoof by windowed PWAs and presence of a title bar. In other words, a user could install a PWA that after launch looks like Chrome for example thus spoofing the user to enter privacy-sensitive data. source

Ultimately, we have to put users first and ensure that we have appropriate mitigations in place to stop potential bad actors. One of those mitigations is the fixed overlay that helps to differentiate a PWA with window controls overlay from the browser itself.

[ghost]

Hi @amandabaker,

Thanks for raising this concern about impersonation of the browser. This is indeed an issue potentially found in three browser features:

• Fullscreen Mode
• Picture-In-Picture V2 with custom HTML elements
• Window controls overlay (this proposal)

It indicates that it is not sufficient to rely on visual cues in the UI for security features. The thing to note is that a malicious website may not need to be pixel perfect to abuse users, as it primarily exploits their inattention. Therefore, a window control overlay does not offer more security than custom window controls designed from scratch in HTML and CSS.

The main security issue is indeed the potential misuse of iframes and portals, allowing the impersonation of the browser by a malicious website acting as a middleman spying onto the users. That's why one is suggesting several security features for the frameless media feature that one is proposing:

Security model for frameless media feature

1. The frameless media feature would use the "Trusted Web Activity" security model, and by default, it would disallow the use of third-party domains in iframes and web portals, making it impossible to impersonate the browser into a webpage. Third-party domains in iframes and portals could be allowed only when the hosting page is localhost.
2. At load time, the browser would check the presence of the frameless media query into the active stylesheet as well as the visibility of the custom window controls into DOM. If the controls are not available, the browser would fall back to the default browser display mode, or to the "display-override" mode.

This way, the frameless webpage would not depend on the attention of the user to visual cues into the UI in order to be secure and would prevent malicious websites to block the user into the page with no window controls.

[ghost]

@malvoz, @amandabaker, you have to bear in mind that PWA is not a standard and that Mozilla already dropped support for it. So PWA is a dead end for web developers wanting to build desktop-like applications with browser technologies.

That's why I think that a frameless media feature in CSS would be more universal than PWAs, which are very browser specific presently.

[bathos]

It’s true that PWA describes design patterns and a constellation of related APIs rather than a standard. Given those facts, it’s hard to speak about dropping support; FF has and continues to support many APIs associated with PWAs. I assume you’re referring to Mozilla deciding not to pursue installability and standalone mode, though (neither of which they supported in the first place unless you count tentative stuff you had to toggle flags to access).

In any case, the dead end claim seems strange. Plenty of us are working on PWAs with desktop users* right now to great effect and Google and MS’s dedication coupled with dev momentum is the reason you’re seeing cool new APIs like this one popping up. Even the Safari situation has been improving a bit :) While I would love for Firefox to support installation and standalone, they cover everything critical and I’m happy to say our app still works well there. That’s the “progressive” part of PWA — it’s supposed to be flexible to a variety of circumstances, that’s one of the goals.

(It’s probably also worth pointing out that there’s no way Firefox is gonna start letting random sites decorate the window frame of the browser itself? In the absence of standalone mode, it’s hard to imagine this feature making sense, but maybe I’ve misunderstood what you want to see happen here.)

* Our own app’s usage is about 50/50 desktop/mobile currently.

[ghost]

@bathos, thanks for the input. What I meant, and as you said, is that some major browsers like firefox do not support "installation" and "standalone" windows (described in PWA patterns), so it is not actually possible to develop an universal (cross-browser) application with the look & feel of a native desktop application by using the PWA patterns. Firefox doesn't even support Filesystem Access API, so it is even more difficult to build an application with access to local resources in some browsers. In consequence, users are locked into what offer the different browsers.

If softwares like Electron, NW.js, CEF, or webview runtimes, etc. exist and are so popular (though they are very inconvenient and a nightmare to build and distribute), it is only because browser manufacturers didn't implement ways to safely and easily bypass the browser sandbox, and it is very unfortunate, as it is in definitive what both developers and users want to achieve in using all those softwares.

Yes, users want the security of the browser sandbox for browsing the internet, but they also want to be able to use applications with desktop functionality that they could trust and which could bypass the sandbox security according to fine-grained permissions. If the users could transparently grant permissions beyond the sandbox to an application in the context of digitally signed resources that they could trust, it would be way better for the web and computing in general.

The W3C should be aware that unbranded, customizable and advanced desktop functionality is what both developers and users want to be able to use, directly from within the browser platform, that is, without the vendor browser UI or the browser brand interfering in the way, because the web, intrinsically, does not belong to browser manufacturers nor should impose to users a technological lock-in.

The web is meant to be free, neutral and an universal technology that should be able to interface with the local machine when the users want it, without arbitrary barriers or lock-in imposed by the browsers manufacturers consequently to a limited understanding of the web and of what developers and users really want to achieve with their computer, with the convenience of the web platform.

[ghost]

@bathos, I suspect that this technological lock-in imposed to web users by browser manufacturers in the name of security (sandbox) is in fact a commercial strategy that is at the origin of the "cloud" platform. Big money is made by cloud enterprises just because users can not use their local machine with web technology, so they force them to put all their documents and information in the cloud, instead of just using the local machine from the browser. It is strange though that an open-source browser like Firefox which supposedly has no financial interest in preventing the users to interface with their local machine from within the browser also happens to be amongst the ones who actively do not promote the development of features removing those limitations. Perhaps donators to the Mozilla Foundation also happen to be cloud entreprenors or companies. Yes, indeed... according to wikipedia, "In 2006, the Mozilla Foundation received US$66.8 million in revenues, of which US$61.5 million is attributed to "search royalties" from Google"...

[bathos]

I think it’s good to consider the motivations of large business interests with the skepticism dial cranked up high, but I’m not sure I follow your line of reasoning. The Project Fugu initiative began at Google and they’ve invested in it heavily. Its express goal is to ensure the web’s capability set doesn’t stagnate or prevent application development. The File System Access API is an example of their work. If Google were opposed to it, I don’t think they would have developed the standard and I don’t think they would have then implemented it in Chromium. It would be awfully odd for them to be paying Firefox to not implement it :)

Some of Google’s own demos, like Squoosh (which is a legit useful app, a bit more than a demo), emphasize offline capability. With service workers and (for better or worse...) IndexedDB it’s possible to create really rich fully-offline experiences. (Those can be made a bit more slick with standalone installation, but they also don’t require it. After visiting Squoosh, try disabling your internet connection and reloading. It’ll just ... keep working!)

In general, I think it’s in Google’s business interests to promote the web and expand its capabilities. I’m not suggesting this is an altruistic endeavor, but I suspect they’re more in agreement with you about where things ought to head than not. When MS got heavily involved in the PWA world, emphasis on capabilities which enhance desktop experiences specifically seemed to increase accordingly. At the moment, whether one is cynical about these big players or not, I think they’re pretty much “on our side”.

The web’s security model does demand sandboxing. So does, say, the iOS store’s security model. I don’t think this is specific to the web and I would say it’s a good thing in both cases — in fact, one of the reasons I prefer PWAs in general when I have a choice (e.g. Spotify native vs PWA) is that I can be more confident about what capabilities and permissions have been given.

There’s contentiousness about the idea that the act of installation would or should imply any implicit permission granting for “minor” permissions that an uninstalled app would have to expressly request. It seems like folks are generally agreed though that it shouldn’t, which seems to fit with what you want: equal footing for uninstalled and installed experiences. The connection between installation and the overlay API doesn’t seem to stem from a specific intention to only permit usage in an installed app. Rather the API just wouldn’t make sense in the absence of standalone mode, and at the moment, standalone mode is specific to installed apps. It seems then like the issue is whether or not standalone mode should be limited to installed apps.

Perhaps expanding it to uninstalled apps would be possible following the element.requestFullscreen() model. I think it would likely have to be a window API instead — window.requestStandalone() — but it doesn’t seem implausible to me that such a thing could succeed as a proposal in its own right. If such an API existed, then I would expect the window controls overlay CSS API would kinda just come with it for free, since it’s a facet of standalone mode rather than a facet of installed apps and at the moment the two conditions happen to only be able to occur together. This could potentially be brought up as a new proposal, though I haven’t researched whether it’s already been discussed elsewhere.

As for the other part of what you’re looking for — disabling the system window controls overlay entirely — I have a hard time imagining this being deemed acceptable by any browser, OS, or security working group (worth noting, I think that those security groups are unusually neutral parties in general). It’s also something that a lot of users could find disorienting, and a core principle of the web is that the most important constituency is its end users, many of whom probably wish native apps couldn’t do this either.

[ghost]

@bathos, what matters in definitive for web technologies is not Google's voice or FF's voice independently, but the W3C voice which sets the standards for the world wide web. And in the W3C there actually are representatives of different browser manufacturers and companies, including those who have vested interests into the actual internet model, which is clearly broken and serving not the end-users but the companies doing business on the internet upon technological limitations imposed by design to the end-users.

So even if Google or Microsoft develop a few features in the sense of more desktop functionalities, but if other browser manufacturers or W3C members are not favorable to them, then it won't make it into the standards, and it won't change the intrinsically flawed design of the internet. There are many stories in various industries of tacit agreements between companies that actually serve the financial interests of those companies but not the end-users of the products. The web is surely not exception, and I'm pretty sure that the W3C has become an organization in which vested interests from different companies have gained over the years a lot of momentum regarding to the decisions.

It is a false statement to say that the web is actually secure or private for users. For example, nowadays spywares called "analytics" are directly installed into most webpages, letting no control over what information within the webpage gets send to unknown organizations into the cloud, who will sell or use that information without the web user's consent. But web users have no other choice, as within this model, they can not have trusted services that could safely interact with their local machines and resources, while keeping strict and transparent boundaries with the internet. The internet security model is broken because it creates a strict boundary between the computer and the browser instead of creating it between the browser and the internet.

So, I'm not talking of indexeddb or service workers, which still are confined to the browser sandbox, I'm talking about the ability for the end-users to use web technologies for their own interest, by interacting with local resources, local programs and the local machine, and to have a security model allowing that, and which doesn't lock them into the cloud. Of course, we need sandboxing when browsing the internet, but we also need to be able to bypass the sandboxing when one needs to and to grant fine-grained permissions to web applications that we can trust, in the context of a transparent security model. Presently web applications can absolutely do what they want with what the web user do within the browser, while the web-user can not do anything with its own local machine. This means that the web technology does not belong to the web user but to web companies serving their own interests while proposing services that are locked into the cloud. It is not an acceptable situation for the web.

It is not disorienting for a user to install an app into their machine, so i do not see why it would be disorienting to install a web app which would have its own window controls in HTML/CSS via a frameless display-mode, and which could also use the default ones if the CSS is not overriden. Also, I do not see why, for a web app that one would install from the browser, it would be less secure than any other desktop app installed onto the computer, to have desktop functionalities. It is just a matter of having a security model that is transparent to the user, who would know what permissions are required to be granted for the application.

It would even be more secure than what the browser manufacturers are actually doing for their own product, which do not explicitely and transparently informs the users about which information gets sent to the internet by the browser application itself.

[ghost]

Another option is to have native, CSS customizable and extendable HTML tags for, "minimize", "maximize", "restore" and "close" window controls:

<window-close/>
<window-minimize/>
<window-maximize/>
<window-restore/>

[Acmion]

I agree that a frameless window would be a great addition. At the very least developers should be able to somehow customize the native controls to be able to provide a clean and consistent UI.

See this illustration (Google Chrome, but similar issues exist with at least Edge.):

• The button shapes are inconsistent. Regardless of button type.
• The hover colors are inconsistent. Regardless of button type.
• The icons are inconsistent. Regardless of button type.
• Inconsistent margins appear. Regardless of button type.
• The native buttons are not of full height.
• Unwanted buttons may appear (see the translation button).

Sure, the custom buttons can be made to conform to the style of either the chrome buttons or the native buttons, but not with both.

By allowing devs to style these elements by themselves, most of these issues could already be avoided.