johnwilander commented 6 years ago

Details to be discussed in the W3C Privacy CG

We've moved this to the W3C Privacy CG where you can file individual issues on the things you want to discuss: https://github.com/privacycg/storage-access

Original issue

Hi! John Wilander from WebKit here. We hope this can extend existing specifications rather than create some whole new spec. It was originally filed under whatwg/dom but I was advised to move it here.

Storage Access API

Problem

Tl;dr: Browsers that block access to third-party cookies break authenticated embeds such as commenting widgets and subscribed video services.

In the context of cross-origin resource loads, cookies are popularly referred to as third-party cookies. In reality, these cookies are often the same as the first-party cookies so what we really mean with third-party cookies is access to first-party cookies in a third-party context.

A browser may have rules for third-party cookies that go beyond cookie policy rules such as scheme, host, path, secure attribute etc. These additional rules may be:

Third-parties aren't allowed to set cookies,
Third-parties have their cookies partitioned or double-keyed, or
Third-parties have no cookie access.

However, certain services are intended to be embedded as third-party content and need access to first-party cookies for authentication. Examples are commenting widgets, video embeds, payment provider integration, document embeds, and social media action widgets. These break if the third-party content has no access to its first-party cookies.

The same problem exists for other kinds of storage such as IndexedDB and LocalStorage, except they are not tied to the HTTP protocol and are typically not used for authentication purposes. From here on we will refer to cookies except when the distinction between cookies and other storage makes sense.

Proposed Solution

Tl;dr: A new API with which cross-origin iframes can request access to their first-party cookies when processing a user gesture such as a tap or a click. This allows third-party embeds to authenticate on user interaction.

We propose two new functions on the document:

partial interface Document {
    Promise<bool> hasStorageAccess();
    Promise<void> requestStorageAccess();
};

The reasons these are on the document is that 1) storage access is granted to the particular document (see Access Removal) and 2) it changes document.cookie.

hasStorageAccess() can be called at any time to check whether access is already granted and it doesn't require user interaction.

requestStorageAccess() should only be called on user interaction such as a tap or a click. It will check a set of rules and grant access if the rules are fulfilled. Access to first-party cookies in the given iframe can be assumed if the returned promise resolves. From that point, any sub resource load in the iframe will have first-party cookies sent and incoming cookies will be set in the first-party cookie jar.

Note that no other third-party resources on that webpage are affected by the storage access status of an individual iframe.

Algorithm for requestStorageAccess()

If the document already has been granted access, resolve.
If the document has a null origin, reject.
If the document's frame is the main frame, resolve.
If the sub frame's origin is equal to the main frame's, resolve.
If the sub frame is not sandboxed, skip to step 7.
If the sub frame doesn't have the token "allow-storage-access-by-user-activation", reject.
If the sub frame's parent frame is not the top frame, reject.
If the browser is not processing a user gesture, reject.
Check any additional rules that the browser has. Examples: Whitelists, blacklists, on-device classification, user settings, anti-clickjacking heuristics, or prompting the user for explicit permission. Reject if some rule is not fulfilled.
Grant the document access to cookies and store that fact for the purposes of future calls to hasStorageAccess() and requestStorageAccess().

Access Removal

Storage access is granted for the life of the document and as long as the document's frame is attached to the DOM. This means:

Access is removed when the sub frame navigates.
Access is removed when the sub frame is detached from the DOM.
Access is removed when the top frame navigates.
Access is removed when the webpage goes away, such as a tab close.

In addition, the browser may decide to remove access on a timeout basis or on some dedicated user action such as switching cookie policy.

WebKit Specifics

WebKit's implementation of Storage Access API will be available in Safari Technology Preview soon and on by default. It only covers cookie access, i.e. no other storage mechanisms and the partitioning of them is affected by a call to requestStorageAccess() at this point.

ppeg34 commented 6 years ago

Can you please elaborate on what is meant by...

the partitioning of them is affected by a call to requestStorageAccess() at this point.

How is the partitioning of the cookies affected by calling requestStorageAccess()? Thanks in advance.

johnwilander commented 6 years ago

Can you please elaborate on what is meant by...

the partitioning of them is affected by a call to requestStorageAccess() at this point.

How is the partitioning of the cookies affected by calling requestStorageAccess()? Thanks in advance.

That is, as you can see, a WebKit-specific detail and not really part of what we propose. But sure.

Let's say example.com has an iframe from social.org and social.org's cookies are partitioned under example.com. If the social.org iframe calls document.requestStorageAccess() upon a user gesture, and the rules say that access should be granted, the returned promise will resolve.

From that point, social.org's cookies are no longer partitioned under example.com in that iframe. Instead it has access to whatever cookies it would have if the user went directly to social.org as a first-party website.

Access goes back to its partitioned state when social.org's document in that iframe goes away or its frame is detached from the DOM.

The same would apply to any other partitioned storage that the browser supports in Storage Access API.

johnwilander commented 6 years ago

Small note on "access to whatever cookies it would have if the user went directly to social.org as a first-party website": That obviously wouldn't apply to SameSite cookies. WebKit don't support those yet, but I wanted to mention that exception.

dlongley commented 6 years ago

We ran into the partitioned storage problem when writing polyfills for the Payment Handler API and Credential Handler APIs and running them on Safari. These needed to load content via a third-party iframe that maintained previously registered payment and credential handlers that were kept track of via localStorage/IndexedDB.

When the relying party website (that loaded the polyfill iframe) attempted to request payment or credentials on Safari, the iframe would fail to see what the user previously registered -- and there was no way to fulfill the request. It sounds like this API would address the problem, which would be fantastic.

jeisinger commented 6 years ago

As I mentioned at TPAC, Chrome offers granular cookie & site data controls and UI that allow for users configuring this without any special APIs, so I'm a bit surprised that we'd need to change the spec for something that other browsers already can offer.

I'm also not sure why we'd need a separate API for this as opposed to using the permissions API.

johnwilander commented 6 years ago

Are you saying Chrome has cookie controls that allow the user to block or partition all or some cross-origin resources’ cookies, and then allow those subresources cookie access on a per-iframe or per page basis?

mikewest commented 6 years ago

As I mentioned at TPAC, Chrome offers granular cookie & site data controls and UI that allow for users configuring this without any special APIs, so I'm a bit surprised that we'd need to change the spec for something that other browsers already can offer.

Browsers can offer the functionality natively, but it might be reasonable to give sites the ability to ask for more permission than they're granted as-configured. In the same way that the permission API gives sites the ability to ask for geolocation, even when the browser is configured to deny-by-default, the problem statement above posits that it might be reasonable to allow sites to ask for permission to access locally stored data, even if the existing configuration would deny that to them.

If we model storage as a permission, as @jyasskin suggested elsewhere, it doesn't seem out of the question to me that developers could call something like navigator.permissions.query({name:'first-party-storage'})) to determine whether their first-party storage is available (perhaps behind a prompt), and call something like navigator.storage.requestFirstParty() (or something similar, but navigator.storage feels like a better spot than Document if we mean for this to cover more than cookies) to cause the prompt.

Also, as @annevk noted in the DOM bug, and @raymeskhoury has noted in a Permission Delegation proposal prompting from a third-party context is hard. It's interesting to think about some of the secondary consequences of modeling this as a permission, if some browsers are shifting to a world where we deny third-party permissions by default and allow/require the parent to delegate. Requiring allow on a frame seems pretty reasonable when we're talking about additional privileges beyond what developers have traditionally expected by default (geolocation, etc). I imagine the breakage (and other side-effects, like the tacit encouragement for third-party scripts to execute in their parents' contexts) would be greater if we shifted things like storage into a deny-by-default mode.

johnwilander commented 6 years ago

Mike, do you still think the API should be on navigator given that any access is scoped to the document?

othermaciej commented 6 years ago

To explain the context for this a bit more, I'd like to explain some things about Safari's current storage policy for third-party contexts, and also some model use cases for this proposed storage access API. I think some of the suggestions here may be due to lack of clarity about these things.

Safari's third-party storage policy

Safari has long partitioned or denied all types of storage except cookies, including even incidental storage like the cache, to prevent tracking via so-called super cookies like evercookie. Partitioning means keying based on both the origin of the resource and the top-level document's origin, effectively making third-party contexts see completely different storage universes on each site they were embedded into.

Cookies were an exception, for web compatibility reasons. Cookies were partially blocked via our long-standing policy that prevents setting new third-party cookies. But if a third-party cookie already existed, third-party origins could read and set it.

In 2017, we extended this to also partition cookies in third-party contexts (with a few limited exceptions). As a result, most content has access to only a partitioned view of its storage including cookies.

The reason for all this is privacy. We don't want embedded content to be able to passively track users across the web. We're most interested in preventing passive tracking, not necessarily incidental tracking when a user chooses to engage with a cross-site embed.

Use cases

YouTube videos are commonly embedded in third-party sites. When the user plays a YouTube video embedded in another site, it would be good to respect their global YouTube preferences, for example caption settings. It would also be good if YouTube Red subscribers were not shown ads, as per the terms of their subscription. But if YouTube's cookies are partitioned, the video player frame cannot see the user's first party
A number of sites have Facebook "like" buttons. These buttons register that a user has "liked" the article with their Facebook account but without leaving the page. If Facebook's cookies are partitioned, then FB doesn't know who the user is and has to navigate or pop up a window to ask. (Facebook "share" buttons and Twitter "tweet this" buttons are unaffected, because they always open a new first-party page on the relevant site to complete the action).

How the Storage Access API helps with cases like this

When the user clicks on a YouTube video to play it, or a Facebook like button to like an article, the embedded content may requestStorageAccess(). The API can only be called within the scope of a user action. Although prompting is allowed by our proposal, we expect in most cases we will just grant the permission without prompting, as the user action is sufficient. We will prompt only if we suspect some funny business is going on, and even then we may just choose to always deny. The API is asynchronous just to enable the possibility of prompting. First-party storage access is allowed only in that frame, and only until it is navigated or removed from the document.

In the examples above, this would allow YouTube to respect the user's settings, and Facebook to know who they are to register the "like". However, it would not enable passive tracking. The user has to click every time to expose first-party state.

Why some other ideas in this issue would not work

Manual fine-grained cookie settings do not solve this problem. They do not enable the user to make Facebook "like" buttons and YouTube video preferences work, but without enabling these sites to passively track them even on pages where they don't engage with these features. IF the user allowed third-party cookies for youtube.com or facebook.com on all domains, this would enable passive tracking. If the user allowed third-party cookies for these domains when embedded in particular other domains, they'd have to predict exactly the sites where they will watch youtube videos or click like buttons, which is not reasonable.
Treating this as a generic permission that is delegated via Feature Policy. We don't want sites to be able to grant their iframes first-party storage access by adding markup to the iframe. This would enable passive tracking even when the user does not engage. And embed providers would be incentivized to add this permission to their standard embed markup. The idea is that embeds only get to identify the user when the user engages with them.

How this might apply to other browsers

Other browsers, including Mozilla and Brave, have expressed interest in denying storage to third-party origins either by default or in optional modes. We believe this API could allow embedded gadgets to function correctly under these types of policies.

dlongley commented 6 years ago

@othermaciej,

First-party storage access is allowed only in that frame, and only until it is navigated or removed from the document.

Would the choice be remembered on the same site in the future without user interaction? I imagine not -- which has me wondering how this would impact embedded content that only loads after the user has engaged in some activity on the top-level domain.

For example, consider a payment handler polyfill that shows the user their registered payment handlers for selection after they've hit a pay button on the top level domain.

With storage partitioning enabled and this new API, the registered payment handlers would be inaccessible (being stored in localStorage/IndexedDB by a third-party polyfill domain) until the user clicks on the embedded content. This may be acceptable the first time it happens, by adding a one-time prompt requesting that the user enable payment on the site. But introducing an extra click to the payment process every time they buy something from the same site would be unfortunate.

It would also be unfortunate to force the top-level domain to use a specialized payment button that is served from the third-party polyfill domain as a mitigation to this problem. One of the advantages of standardizing payment on the Web is to avoid the Nascar problem and forcing top-level domains to use specialized buttons.

I wonder if some kind of one-time use "delegated user interaction object" could be passed to the iframe via postMessage if specifying delegated permissions in the iframe markup is considered unacceptable. That sort of primitive may also be helpful/reusable in other situations on the Web platform, particularly those where a Promise must resolve prior to "consuming" a user interaction to call some guarded API.

othermaciej commented 6 years ago

Would the choice be remembered on the same site in the future without user interaction?

No.

I imagine not -- which has me wondering how this would impact embedded content that only loads after the user has engaged in some activity on the top-level domain.

I don't think it would help in such cases (unless the user clicks again on the embedded content).

For example, consider a payment handler polyfill that shows the user their registered payment handlers for selection after they've hit a pay button on the top level domain. With storage partitioning enabled and this new API, the registered payment handlers would be inaccessible (being stored in localStorage/IndexedDB by a third-party polyfill domain) until the user clicks on the embedded content.

Do you know a real site or embedded gadget that actually works this way? It would specifically need to:

Load an iframe after the user initially clicks.
Show a set of payment providers based on info specific to the user, with that set being accessed from the iframe.
Not involve any clicks prior to selecting a payment provider in its normal flow.
Not make the initial payment button itself an iframe.

I don't know of any setups like this, but if there are, we can certainly think about opportunities to help them in the face of limitations on third-party storage.

dlongley commented 6 years ago

@othermaciej,

Other than the payment handler and credential handler polyfills -- I don't know of any at the moment. There may be some some third-party payment providers that provide this kind of flow, but I'm unaware of them.

This is the payment handler polyfill demo that was presented at TPAC that I'm talking about (note that this won't run in Safari because of this very issue, but will run in Chrome, Firefox, and Edge):

You install a payment handler and add payment instruments (credit cards) here at a "wallet" website:

https://payment-handler.demo.digitalbazaar.com/

And then you use the registered handler here via a payment instrument of your choice here at a "merchant" website:

https://payment-merchant.demo.digitalbazaar.com/

This follows the flow you described above.

othermaciej commented 6 years ago

It does seem like that demo would need to be modified even with this new API in place. It will either need an extra click to look up the default payment method, or the buy button itself needs to be an iframe.

Since this demo is not (as far as I know) in wide actual use, I am not sure it should be a priority to design around its requirements rather than to look at ways to modify the demo.

mikewest commented 6 years ago

Thanks, @johnwilander and @othermaciej for the detailed background. That's helpful information. I'll pull out a few points below:

Mike, do you still think the API should be on navigator given that any access is scoped to the document?

If we model things as a permission, then I think hooking into the existing navigator.permissions and navigator.storage makes more sense than adding one-off methods to Document. I'd also push back a bit on the notion that "access is scoped to the document", as, at least for cookies, HttpOnly cookies wouldn't be accessible to the document, but instead sent on requests to the origin via HTTP requests which initiate from the document (or, presumably, from its parent?). Ideally, those cookies would never be exposed to the document context.

When the user clicks on a YouTube video to play it, or a Facebook like button to like an article, the embedded content may requestStorageAccess(). The API can only be called within the scope of a user action.

I think this addresses some of the use cases you noted above, while partially addressing others. Video sites are good examples of embeds that need to make decisions about what content to show a given user prior user interaction in order to comply with a user's preferences regarding mature content (Vimeo's viewing preferences, YouTube's restricted mode, etc.). It seems like this mechanism might push some of those sites into a client-side rendering model whereby content is locked behind a clickthrough that calls out to the mechanism y'all are proposing before rendering anything. That seems doable from a technical perspective, but would have performance (and aesthetic!) impacts on users.

Although prompting is allowed by our proposal, we expect in most cases we will just grant the permission without prompting, as the user action is sufficient. We will prompt only if we suspect some funny business is going on, and even then we may just choose to always deny. The API is asynchronous just to enable the possibility of prompting. First-party storage access is allowed only in that frame, and only until it is navigated or removed from the document.

All of these restrictions would be compatible with treating storage access as a permission that the browser makes decisions about whether to grant or restrict, with or without user interaction. The scoping and persistence is a bit different than we've done with other permissions in the past (and different still from what Chrome folks are proposing), but the core notion of asking first, and then doing, seems like a reasonable fit.

In the examples above, this would allow YouTube to respect the user's settings, and Facebook to know who they are to register the "like". However, it would not enable passive tracking. The user has to click every time to expose first-party state.

It's somewhat tangential to the core proposal, but I'd suggest that y'all will want to consider whether or not the user gesture is consumed by the call to obtain first-party storage. I know that Chrome's gesture implementation has sometimes caused developers some annoyance by disallowing combinations of actions with a single gesture. I can imagine that applying here as well: perhaps a widget would ask for credentials on click, and use them to make a decision about whether to call window.open() (which I believe Safari also gates on a gesture requirement).

Manual fine-grained cookie settings do not solve this problem. They do not enable the user to make Facebook "like" buttons and YouTube video preferences work, but without enabling these sites to passively track them even on pages where they don't engage with these features.

Chrome's content settings are certainly capable of making this distinction between "B in A" and "B in C" (see the Primary and Secondary Patterns section of the chrome.contentSettings extension API docs, for example), and allows the user to do so in certain cases via the blocked-cookie UX in our omnibox. I suspect we could do a better job exposing those granular options to the user in our settings UI, but that in some ways comes back to third-party contexts being difficult to expose to users in a way that they can be expected to generally understand.

IF the user allowed third-party cookies for youtube.com or facebook.com on all domains, this would enable passive tracking. If the user allowed third-party cookies for these domains when embedded in particular other domains, they'd have to predict exactly the sites where they will watch youtube videos or click like buttons, which is not reasonable.

I know that some users (like @jeisinger above :) ) have developed workflows whereby they toggle Chrome's "Block third-party cookies and site data" option, and then make use of the omnibox UX and the reload button to fix sites after visiting them. It seems to me that that's not a terrible model.

Treating this as a generic permission that is delegated via Feature Policy. We don't want sites to be able to grant their iframes first-party storage access by adding markup to the iframe. This would enable passive tracking even when the user does not engage. And embed providers would be incentivized to add this permission to their standard embed markup. The idea is that embeds only get to identify the user when the user engages with them.

Another way of looking at the same mechanism would be that the top-level document would be able to use Feature Policy to deny access to certain permissions to its children by fiat, the embeddee's wishes notwithstanding.

Moreover, even in a fully realized delegation model, the browser remains in a position of mediating the permission requests, and could certainly make a decision about whether to grant permissions on the basis of information to which the page isn't privy. The permission model means that the request would fall into a well-paved API path for developers, not that you'd be locked into adhering to a page's desires without the potential for override.

Other browsers, including Mozilla and Brave, have expressed interest in denying storage to third-party origins either by default or in optional modes. We believe this API could allow embedded gadgets to function correctly under these types of policies.

/cc @tanvihacks, @dveditz, @diracdeltas from those browsers for feedback. Also tagging in @patrickkettner to get Edge in the loop.

diracdeltas commented 6 years ago

Hi, Yan from Brave here! Brave does indeed block 3rd party storage (cookies, localStorage, etc.) and referrer by default.

Although prompting is allowed by our proposal, we expect in most cases we will just grant the permission without prompting, as the user action is sufficient.

We (Brave) would almost certainly want to prompt by default with a 'never prompt again' option in the permissions UX. I think this API would be useful but we would want to be careful to not make our current cookie restrictions more lax without alerting the user.

othermaciej commented 6 years ago

@diracdeltas The API is designed to allow all the prompting you want (since it's async) so it would be easy to fulfill it with a prompt-always (or prompt-once-per-origin-pair) policy.

hillbrad commented 6 years ago

Yes, it would be great to allow for never prompting again. Many integrations that involve data sharing are moving towards a more granular model of permissions, with an emphasis on prompting only in context and as needed, rather than asking for all permissions that might ever be needed up front. The more friction the browser adds to this, the less likely sites are to adopt these granular and just-in-time models.

johnwilander commented 6 years ago

From today's W3C WebAppSec call, as I interpreted it:

Mozilla is “cautiously interested” in implementing. They will monitor the uptake of the API in Safari and if it seems to work, they will try it out.
Edge said their status is similar to Mozilla’s.
Chrome has no plans but might implement in Chromium for other ports, especially if this gets standardized.

diracdeltas commented 6 years ago

@johnwilander For now, Brave's status is that we would use this if it were already implemented in Chromium. We could be interested in implementing it independent of Chromium depending on adoption.

mikewest commented 6 years ago

Chrome has no plans but might implement in Chromium for other ports, especially if this gets standardized.

What I hope I suggested on the call was that I don't think Chrome has a concrete interest in implementing this, but that if a Chromium port (like Brave) wanted to upstream a patch to reduce their maintenance burden, I'd be happy to help them do so.

The WebAppSec discussion's draft minutes are available at https://www.w3.org/2018/01/17-webappsec-minutes.html#item04 if folks here are interested in the points raised there. @dveditz and I re-raised the suggestion that this might fit into the Permissions API rather than building a new one-off on Document, and @jyasskin suggested that if the Permissions API spec contains restrictions on the permission model that make it incompatible with this usage, he'd see it as a bug and fix it.

annevk commented 6 years ago

The "has" method can be modeled as a permission I think. "Request" not so much.

FWIW, there's quite a few people within Mozilla that have a problem with prompts on behalf of third-parties and I'm not really sure how to reconcile that position with this API.

mikewest commented 6 years ago

The "has" method can be modeled as a permission I think. "Request" not so much.

I agree. One of the earlier suggestions in this thread was to tie the request bit to a new method on navigator.storage rather than Document.

FWIW, there's quite a few people within Mozilla that have a problem with prompts on behalf of third-parties and I'm not really sure how to reconcile that position with this API.

That's also something Chrome folks are reluctant to do more of.

johnwilander commented 6 years ago

As for prompting, it’s optional in the proposal. Are you saying you’re against optional prompting too? Brave expressed that they’ll definitely prompt. We want the ability if we see abuse of the API down the road.

annevk commented 6 years ago

Without prompt it's just the user clicking/touching somewhere in the frame. So if we assume this'll be abused, we'd have to prompt, at which point we have a problem.

johnwilander commented 6 years ago

As listed in the proposal, there are other ways to fight abuse, such as heuristics to fight clickjacking or on-device statistical monitoring of API use.

jyasskin commented 6 years ago

IIUC, Safari wants to treat this a bit like Chrome treats the vibrate API, where user activation in the frame is enough to grant permission. However, the effect is to swap out all of that frame's storage, which the frame might not actually want to do. Maybe the frame can get along fine with partitioned storage as long as that storage is consistent for the life of the frame. So Safari needs the frame to call a function (document.requestStorageAccess() or navigator.storage.requestFirstParty()) to opt into the storage swap.

Making that function return a Promise seems like the only sensible signature for it: in a multiprocess browser, especially after Spectre, swapping cookies and localstorage needs an IPC, so it can't be a synchronous JS call. The Promise return also happens allows the browser to ask the user's permission if it really needs to, either because it's extra-cautious like Brave or because we've found an abuse vector in the future. Yes, that permission request is hard to design, so maybe the non-Brave browsers will never find it worth showing, but the API shape should still be to return a Promise.

jyasskin commented 6 years ago

One argument against using permissions.query() is that query() returns denied/prompt/granted, and the non-Brave case will be that the "prompt" state actually doesn't show a prompt.

mikewest commented 6 years ago

... the API shape should still be to return a Promise.

I agree with this entirely, the shape of the API @johnwilander is proposing seems right to me for the functionality.

One argument against using permissions.query() is that query() returns denied/prompt/granted, and the non-Brave case will be that the "prompt" state actually doesn't show a prompt.

Based on Macej's comment above, it sounds like developers will need to deal with a potential prompt based on in-browser heuristics, and with differing behaviors cross-vendor. In that case, prompt still seems like the right answer, as they'll only be able to call the request-side of the API in cases where they'd be fine with a prompt appearing in front of the user.

wanderview commented 6 years ago

Without a prompt, is there any incentive for 3rd party ad/tracking frames to not spam requestStorageAccess() at every opportunity?

If the answer is that heuristics will just deal with it, why even have the requestStorageAccess() method? It seems these heuristics could just grant permission to begin with.

Sorry if I'm confused.

johnwilander commented 6 years ago

The answer is heuristics and the reason to require an explicit request is to offer the document some control over when its cookie jar changes.

wanderview commented 6 years ago

Hmm, ok.

Just for the record I guess I'll add some implementation concerns I have here. I think we would need to address this stuff before we could possibly implement in gecko.

My big concern is:

In gecko we implement storage partitioning, etc, in our "origin" primitive itself (the nsIPrincipal). Turning off partitioning basically means rebranding then entire page with a different origin. I am pretty concerned that this could not be done in a sane or safe manner without reloading the entire page.

Some other questions/concerns I have:

What happens if the page has a handle to partitioned storage open when requestStorageAccess() is called? For example, an IDB transaction or a Cache API object, etc. Does it then have the ability to read/write both partitioned and non-partitioned storage at the same time? Or do the open ones abort somehow? Or must the browser reject the request for storage access?
Is it possible for a site that has its cookie jar changed to communicate with other windows in the partitioned origin to effectively access both partitioned and un-partitioned storage at the same time? Methods for doing this could be BroadcastChannel, SharedWorker, and ServiceWorker. I know webkit doesn't have the first two, but it will become an issue with ServiceWorker soon. Again, reloading the site completely in the new non-partitioned origin would address this.
Currently we block access to service worker interception and APIs if storage is not permitted for a site. Also, the service worker that controls a page is based on the origin. If a site is granted non-partitioned storage access how does this impact service workers? Does the page become controlled? (I assume not) What if it was controlled in the partitioned origin but not in the un-partitioned origin? We don't have a way to move a page from controlled to uncontrolled. Perhaps this is related to (2), but its a bigger issue than just communicating with another context in the same origin that hasn't been granted permission.

othermaciej commented 6 years ago

@mikewest

Based on Macej's comment above, it sounds like developers will need to deal with a potential prompt based on in-browser heuristics, and with differing behaviors cross-vendor. In that case, prompt still seems like the right answer, as they'll only be able to call the request-side of the API in cases where they'd be fine with a prompt appearing in front of the user.

We don't want to synchronously distinguish the "prompt" and "denied" cases:

It's an extra bit of fingerprinting info.
Sites don't need it in this case: there's no harm in always asking and waiting for the promise to resolve, when the API is available.
We may not be able to answer whether we will prompt without IPC to another process.

If you interpret "prompt" to mean no decision binding on this frame has been made, then we could answer synchronously. However, the permission is scoped only to the lifetime of a single document in a single frame. So query could only return "prompt" in a given frame before the frame calls it, and will always return "denied" or "granted" once it has been called once. So under this interpretation, "prompt" only means "I haven't called this API yet", which doesn't seem like very useful info.

In brief, while reuse is good, using the permissions.query here seems to be shoehorning in a concept that doesn't quite fit.

jyasskin commented 6 years ago

@othermaciej Note that permissions.query() is not synchronous.

othermaciej commented 6 years ago

Ok, my mistake. That makes it less objectionable, but my other points still apply. Is there a valid use case for distinguishing “prompt” from “deny” for this particular request?

mikewest commented 6 years ago

So query could only return "prompt" in a given frame before the frame calls it, and will always return "denied" or "granted" once it has been called once. So under this interpretation, "prompt" only means "I haven't called this API yet", which doesn't seem like very useful info.

Perhaps not for Safari, given your explanations above, but it's not clear to me that other vendors would choose the same constraints. For example, @diracdeltas suggested above that "We (Brave) would almost certainly want to prompt by default with a 'never prompt again' option in the permissions UX.", which would make the distinction between prompt and granted valuable on an ongoing basis.

Is there a valid use case for distinguishing “prompt” from “deny” for this particular request?

In the general case, it's helpful to know that you're just not going to get a given permission. That said, it seems reasonable for a given browser to return prompt rather than deny (and then auto-reject a request) if hiding the true value is important.

sechel commented 6 years ago

We are waiting for this being applied to indexedDB. +1

johnwilander commented 6 years ago

Are you saying you want this API to apply to IndexedDB in Safari as a developer or that you want Safari to make it apply to IndexedDB before you implement the API in another browser?

sechel commented 6 years ago

@johnwilander We develop a web app that handles sensitive user date. We have a security mechanism in place that is based on origin separation between frontend (top-level context) and data storage (iframe on a different origin via postMessage api). Currently we support Chrome and Firefox which allow for 3rd party storage access. We respect the policy of the WebKit team to protect the privacy of the user and prevent tracking by employing partitioning on all 3rd party storage systems. At the same time this is a blocker for us when implementing our system for Safari on Mac and iOS. The Storage Access API seems like the right idea if applied to all storage types. BTW: If Service Workers (and IndexedDB support) are deployed the way they are planed today tracking becomes perfectly possible, since IndexedDB in service workers are not partitioned and can be spawned from any iframe of any origin.

cdumez commented 6 years ago

BTW: If Service Workers (and IndexedDB support) are deployed the way they are planed today tracking becomes perfectly possible, since IndexedDB in service workers are not partitioned and can be spawned from any iframe of any origin.

Did you try in Safari Technology Preview?

Service workers cannot be spawned from any frame from any origin. You can only register a service worker for your origin as per the specification. A service worker can only intercept loads for a given origin.
Safari double-keys service workers based on top-origin / frame origin. A facebook.com iframe under google.com would get a different service worker than a facebook.com top frame.

sechel commented 6 years ago

Did you try in Safari Technology Preview?

Yes and it's exactly as I said. Maybe you should file a report if this is not the intended behaviour.

cdumez commented 6 years ago

Ok, looks like Youenn is working on fixing this bug: https://bugs.webkit.org/show_bug.cgi?id=183496

Thanks for pointing this out.

othermaciej commented 6 years ago

@sechel Indeed, thanks, it's not the intended behavior.

Also, your use case for IndexedDB support for Storage Access API sounds reasonable. We probably can't get that in the first version, but we'd appreciate if you could file the request, along with a description of your use case, at http://bugs.webkit.org/

sechel commented 6 years ago

We probably can't get that in the first version, but we'd appreciate if you could file the request, along with a description of your use case, at http://bugs.webkit.org/

@othermaciej Ok: https://bugs.webkit.org/show_bug.cgi?id=183517

rstoneIDBS commented 6 years ago

Having been recently bitten by this feature (3rd party cookie blocking), I would like to offer a different perspective on the problem relating to extensible enterprise web applications.

We have several enterprise level web applications that are served from different domains. We are now adding integration points between these applications and are using IFRAMEs to accomplish this, however the 3rd party cookie blocking is making this impossible as it breaks cookie based authentication. More importantly, we want to extend these integration points to trusted 3rd party integrators so even if we could host our enterprise apps on the same domain this wouldn't help with applications hosted elsewhere.

The Storage Access API did seem like it would offer a solution, however it's insistence on being triggered via a user interaction is a deal breaker for us as in almost all of our use cases, the IFRAME won't exist until after the user has interacted as we create the IFRAMEs dynamically on demand. In our case, the user requests an action in the main application that results in an IFRAME being created to show information from the second application. The second application is performing a 'silent login' via OAuth, however any cookies created by this process are currently being discarded due to 3rd party cookie blocking.

While I can understand why this approach has been taken, it does seem to be a very broad brush for tackling a specific problem. From my perspective, the 'user permission' option mentioned earlier gives our users a much better experience. Would it not be possible to offer both approaches, i.e. a different IFRAME sandbox token that enables 3rd party cookie storage if the user accepts a permission prompt?

Currently our only option is to advice our Safari users to disable this feature which is obviously far from ideal.

johnwilander commented 6 years ago

While I can understand why this approach has been taken, it does seem to be a very broad brush for tackling a specific problem. From my perspective, the 'user permission' option mentioned earlier gives our users a much better experience. Would it not be possible to offer both approaches, i.e. a different IFRAME sandbox token that enables 3rd party cookie storage if the user accepts a permission prompt?

Intelligent Tracking Prevention is designed to prevent cross-site tracking. Any relaxation of it, such as the Storage Access API, needs to be looked at from a tracker's perspective. If there will be an invisible way to re-enable tracking, for instance by convincing site owners to add a new header or sandbox token, it's a non-starter for us. The user has to be involved.

If we're talking stickiness of storage access, the user probably needs to be prompted. Then comes the question of whether 3rd-parties should be able to prompt the user or not.

At the heart of this lies the binary nature of cookie access. There is no current way of saying "I'd like to use my cookies as third party under these N websites." Either you have access under every website that loads something from you, or you only have access as first party. From an anti-tracking perspective that's the binary question of tracking ability or no tracking ability.

rstoneIDBS commented 6 years ago

@johnwilander - I'm not (at least I wasn't intending to) suggesting a silent mechanism to work around this. For experienced users (which our target users are) a mechanism that allows them to configure their browser to always accept cookies from valid sites (regardless of whether they are 1st or 3rd party) either via a white listing mechanism (such as that offered by Chrome) or via a browser prompt (IE offers this option for 3rd party cookies) would be sufficient.

At the moment, with Safari, our only option (short of some fairly ugly UI hacks) is to tell our users to either use another browser or turn off the cookie blocking functionality entirely.

michael-oneill commented 6 years ago

Could we have 3rd party cookie partitioning (& localStorage etc.) enabled by default via a 1st party header e.g. FP or CSP? So sites could decide whether their embedded 3rd-parties get access to cross-origin cookies. Even better would be to enable it via a source list attribute, so the site could specify a subset of the embedees that are forced to have their cookies siloed, allowing it to offer its visitors granular consent over cross-origin tracking.

whitehatguy commented 6 years ago

Should there be an option for users to allow parties to access their own cookies across different origin domains? By "party", I mean the actual business entity (as identified by the organization in their OV or EV certificate). I know of many larger organizations that own different origin domains for branding purposes and/or for optimization (regional content delivery, etc.) It feels like since this vetted (vetted by a CA that participates in CA Browser forum) information is available to the browser, it could/should be leveraged to ensure unobtrusive experiences (example: not asking the user if Google YouTube can access Google Website cookies) while still protecting privacy.

kushal commented 6 years ago

@johnwilander This is a really elegant solution! Very cool. For our purposes, one last piece that would be quite helpful would be to know if Storage Access is likely to result in cookies being found, so that we can selectively display UI to users who are likely to interact with it, resulting in a cleaner user experience for everyone.

One unintrusive option might be an argument ifPresent to hasStorageAccess that makes it return true if it thinks the result of access would be a no-op anyway, or I'm sure there's a more graceful way to do this.

I can imagine some risk here of it being leaky similar to HSTS, but WebKit's recently-announced supercookie mitigations are promising and similar limitations could be applied here.

(FWIW, either a) allowing users to just grant global permission to certain domains or b) adding some sort of Cookies-present HTTP header in requests where cookies have been elided would work even better for us, but I assume there might be more momentum around extending the existing path here. Maybe there's even a middle ground of allowing users to grant permission for a domain to reval this binary bit of information as they go about the web ("Let sites I visit know that I'm a foo.com user") but requiring interaction before the cookie value itself is released.)

bvlgn commented 6 years ago

I would like to propose that implicit first party storage access would be granted for a site loaded in a iframe of a parent site when both sites share the same (SAN) SSL certificate. So when either the certificates ‘Subject’ or ‘Subject Alternative Name’ match then grant first party access.

Maybe there are other/additional solutions to identify sites as belonging to the same party, e.g. by a new DNS txt records?

This would for example allow things like cross site shopping carts (users not logged in) and silent/auto network logins without being useful for undesired privacy related tracking.

whatwg / html

Proposal: Storage Access API #3338

Details to be discussed in the W3C Privacy CG

Original issue

Storage Access API

Problem

Proposed Solution

Algorithm for requestStorageAccess()

Access Removal

WebKit Specifics

Safari's third-party storage policy

Use cases

How the Storage Access API helps with cases like this

Why some other ideas in this issue would not work

How this might apply to other browsers