Pickling for Async Clipboard API

[snianu]

Ya ya yawm TAG!

I'm requesting a TAG review of Pickling for Async Clipboard API.

Powerful web applications would like to exchange data payloads with web and native applications via the OS clipboard (copy-paste). The existing Web Platform has an API that supports the most popular standardized data types (text, image, rich text) across all platforms. However, this API does not scale to the long tail of specialized formats. In particular, custom formats, non-web-standard formats like TIFF (a large image format), and proprietary formats like .docx (a document format), are not supported by the current Web Platform. Pickling for Async Clipboard API aims to provide a solution to this problem, by letting web applications read and write custom, unsanitized, web-originated payloads using a standardized pickling format.

• Explainer¹ (minimally containing user needs and example code): https://github.com/w3c/editing/blob/gh-pages/docs/clipboard-pickling/explainer.md
• Security and Privacy self-review²: https://github.com/w3c/editing/blob/gh-pages/docs/clipboard-pickling/security-privacy.md
• GitHub repo (if you prefer feedback filed there): N/A
• Primary contacts (and their relationship to the specification):
  • Anupam Snigdha (@snianu), Microsoft
  • Bo Cupp (@BoCupp-Microsoft) Microsoft
  • Darwin Huang (@dway123) Google.
• Organization/project driving the design: Microsoft in collaboration with Google
• External status/issue trackers for this feature (publicly visible, e.g. Chrome Status): https://chromestatus.com/feature/5649558757441536

Further details:

• [X] I have reviewed the TAG's Web Platform Design Principles
• The group where the incubation/design work on this is being done (or is intended to be done in the future): Editing TF
• The group where standardization of this work is intended to be done ("unknown" if not known): Editing TF
• Existing major pieces of multi-stakeholder review or discussion of this design: N/A
• Major unresolved issues with or opposition to this design: N/A
• This work is being funded by: Microsoft

You should also know that...

There have been discussions on Pickling Clipboard API during TPAC and also while discussing Raw Clipboard API. Slides: https://docs.google.com/presentation/d/1_fAgL54D0whQ497G8iL0K2kKpxiWDr3M7gXXSIS76II Editing TF Minutes: https://lists.w3.org/Archives/Public/public-editing-tf/2020Oct/0017.html

We'd prefer the TAG provide feedback as (please delete all but the desired option):

💬 leave review feedback as a comment in this issue and @-notify @snianu @dway123 @BoCupp-Microsoft

[kenchris]

Hi there, @torgo, @LeaVerou and I looked at this today.

One concern - say a web site creates a custom format - then other web sites start to adopt this - then it becomes widespread without sanitization. Is this being restricted to a domain name? People reverse-engineer formats, and this could lead to a lot of unsanitized content in clipboards.

Example, I am Figma and I allow people to paste say JavaScript because it's in my format. Now someone creates an app that can read and write this format and provide some features not in Figma. This could lead users to use both apps and copy/paste between them. But maybe the second app adds malicious JS (or just triggers an untested code path in Figma) which will be executed in figma if people paste it in there.

There doesn't even seem to be a way for sites to see where the pasted content originates (like origin) so do their own sanitation.

[torgo]

Additional comments:

On naming - we've reviewed what you've written in the explainer, but we still feel there could be a lot of confusion about this API especially with non-english speakers. Not sure if there is anything that can be done here.

On privacy - good to see an additional user activation requirement. However, I was under the impression that there is a requirement for user activation on any clipboard read. Is this not the case?

[snianu]

Is this being restricted to a domain name?

We don't want to restrict to a domain because we want native apps to be able to consume contents from sites that opted-in to be using pickling formats, and vice versa. We expect, sites like Figma, GoogleDocs, Office and Libre will need to have access to it, to provide better user experience. Browsers run with the assumption that clipboard contents are untrusted, hence the sanitization of web-exposed formats. As far as pickling formats go, web apps should not assume the content is trusted either and do the due diligence themselves. And this is by design since we want to preserve as much of the original content as possible.

On naming

Pickling formats is not exposed as an API name anywhere. Are you referring to the usage in the explainer and the spec? We could also refer to it as Custom unsanitized mime types with a common naming pattern across browsers"? In the API direct option in the ClipboardItemOptions refers to the custom formats that shouldn't be sanitized. Are we OK with this naming?

On privacy

In Chromium, we do have user activation requirement for execCommand("copy") but I don't think we have one for async clipboard APIs read/write. We only have permission prompts for async clipboard read, but not write. For Pickling APIs we do want to add both user activation and permission prompt for read & write. However there were feedbacks from web developers that if it's gated behind user gesture requirement, then it would be too restrictive and fetching payload (which might be huge) from the server and writing into the clipboard might cause performance issues during copying/pasting. This is sort of implying that the operation is synchronous which defeats the purpose of async clipboard apis.

Adding @dway123 @whsieh @gked @BoCupp-Microsoft to comment more on this.

[torgo]

However there were feedbacks from web developers that if it's gated behind user gesture requirement, then it would be too restrictive and fetching payload (which might be huge) from the server and writing into the clipboard might cause performance issues during copying/pasting.

Is this still under discussion somewhere? I don't understand how this can be reconciled with the need to keep users' clipboard contents private when they visit a web page (which is related to one of our key design principles : https://www.w3.org/TR/design-principles/#safe-to-browse that it must be safe to visit a web page).

[gked]

However there were feedbacks from web developers that if it's gated behind user gesture requirement, then it would be too restrictive and fetching payload (which might be huge) from the server and writing into the clipboard might cause performance issues during copying/pasting.

Is this still under discussion somewhere? I don't understand how this can be reconciled with the need to keep users' clipboard contents private when they visit a web page (which is related to one of our key design principles : https://www.w3.org/TR/design-principles/#safe-to-browse that it must be safe to visit a web page).

There was a discussion on github which can be found here. To be clear, it is not just a random site accessing clipboard content. User will need to give clipboard access to the site through permissions API first. In addition, having a sticky user activation requirement (instead of transient) and an active document in focus would ensure that the webpage doesn't have access to clipboard without user explicitly interacting with the web page at least once.

We are reaching out to web applications so they can weigh-in on this bit as well.

CCing @garykac as FYI

[AbhishekR-Microsoft]

As far as pickling formats go, web apps should not assume the content is trusted either and do the due diligence themselves. And this is by design since we want to preserve as much of the original content as possible.

I am from Office team. Custom formats in Clipboard will benefit our users since it will drastically improve native to web apps copy paste fidelity.

We are good with doing sanitization of custom formats at our end.

[torgo]

@gked

To be clear, it is not just a random site accessing clipboard content. User will need to give clipboard access to the site through permissions API first.

I'm not sure I'm convinced that putting something behind a permission by itself is sufficient for something this powerful and potential privacy infringing, especially considering how we have seen egregious gaming of privacy prompts by bad actors (see here: https://github.com/w3ctag/design-reviews/issues/337#issuecomment-561368571). However, adding a user activation requirement and being the active document in focus (as you've described) may provide additional mitigation. I put this on the agenda for this week's TAG calls to discuss further. Are the risks and mitigations documented in the appropriate security & privacy considerations section? If so, can you point me that way?

[gked]

@gked

To be clear, it is not just a random site accessing clipboard content. User will need to give clipboard access to the site through permissions API first.

I'm not sure I'm convinced that putting something behind a permission by itself is sufficient for something this powerful and potential privacy infringing, especially considering how we have seen egregious gaming of privacy prompts by bad actors (see here: #337 (comment)). However, adding a user activation requirement and being the active document in focus (as you've described) may provide additional mitigation. I put this on the agenda for this week's TAG calls to discuss further. Are the risks and mitigations documented in the appropriate security & privacy considerations section? If so, can you point me that way?

Thank you, looking forward to getting the feedback on this proposal. To your last question, I believe, @snianu has documented privacy and security risks before. Hey @snianu, could you please point @torgo to it?

[snianu]

Here is the link to the security/privacy section in the explainer: https://github.com/dway123/clipboard-pickling/blob/main/explainer.md#privacy-and-security

[torgo]

@gked :

Replaying our discussion in today's TAG plenary call where we discussed this issue. The TAG consensus is that the mitigation described - permission, sticky activation, active document - is not sufficient. This is because it still could lead to unintentional leakage of the information held in the user's clipboard. We don't believe that the user activation as specified contains a notion of a restricted form of activation that carries some kind of intent (a specific intent to paste content). And if this feature should be added to the web, then that kind of activation feature (with intention) should also be added.

One scenario is: you're phished to a page that looks like your bank's web site. that page induces you to agree to the permission through a dark pattern (and also induces some additional interaction in the same way), and then (without an explicit paste command) the web page in question has access to the contents of your clipboard.

Another scenario: you're on a legitimate site and you explicitly paste something into a page which has permission to access your clipboard. Now you go off and copy something else (say a password) into your clipboard which is unrelated to your interaction with this site. And bring the browser window back up. Assuming this site is the active tab, this site would have access to your sensitive information on the clipboard.

We understand the performance issue that's been articulated. Is there another way to address this issue that can afford the user the protection we think they need here.

[gked]

Thanks, @torgo for sharing TAG consensus!

Correct me if I am wrong but its sounds like security scenarios presented are applicable to not just pickling but any format used by async clipboard api. Pickling format in itself doesn't create any additional vectors of attack in this case.

We are reaching out to security reviewers that have reviewed async clipboard api before it was shipped, to better understand rational behind the current API design.

P.S. We took your feedback about "sticky" activation and moved to requiring "transient" activation for reading and writing to the clipboard.

[snianu]

Just a small update. We addressed few security concerns raised by internal security reviews as described below:

1. Changed direct keyword to unsanitized to be more explicit about the content being read/written.
2. Added transient user activation for reading and writing into the clipboard.
3. Format names are also mangled while writing into the clipboard so it's clear to the consumer of those formats that the content is unsanitized and written from the browser. e.g. format names are prefixed with "Web" on Windows

The explainer has been updated and moved to editing working group repo: https://github.com/w3c/editing/blob/gh-pages/docs/clipboard-pickling/explainer.md#pickling-for-async-clipboard-api

[torgo]

@gked sorry for the delayed response but yes that's correct. the scenarios described are applicable to any format used by async clipboard. We did review async clipboard and we did raise this issue however it was our understanding that the feedback had been listened to. And indeed it looks to me from here https://github.com/w3c/clipboard-apis/issues/52 that this issue is still open. Will pick up on that issue.

[snianu]

@torgo Just checking if TAG got a chance to discuss this API. We made some changes in the format naming and how the custom formats are read/written via async read/write methods and would love to get some feedback on this.

[cynthia]

Apologies for the long delay! We discussed this at length during our VF2F, and while we think this would be a great addition to the platform, it's also a scary+open ended feature. The design itself I think makes a lot of sense and we are happy with it, but we'd like to see this being tried out in the wild (say, with some early adopters) and getting their feedback would be useful to know if there are details/features that we potentially missed in the initial design.

Do you think it would make sense to give it a trial run in the wild, and re-open after seeing what kind of feedback comes back? (presumably, also with updates to the design) @snianu

[snianu]

Thank you @cynthia for the feedback! Yes, we do have partners who are very much interested in using this API as it would unlock some of the powerful editing features in their sites(and interop with native apps) and also provide high fidelity copy-paste content in general. For experiments, we do need a resolution from TAG, so is it possible to get some resolution here? I'll reopen the issue & post any feedback we get from our partners if it leads to any significant design changes (probably unlikely).

[cynthia]

Done, please request a re-open when you have more data!