[amp-consent] Fine grain, client-side consent

[mcmd]

Some publishers may want to implement fine grain consent on cookies, discriminating between analytics, marketing, etc. An example of this is what The Guardian is dong today:

Would it be possible to implement this in AMP? The only two ways I can think of are

1. implementing N amp-consents, which wouldn't be practical, as the UI would be messy
2. Using sharedData in amp-aconsent, but this would require defining an endpoint to manage the satate of the consent, which is a high bar for something that was not requirirng an endpoint untill now.

Is there any other way of implementing this fine grain consents? in AMP, client-side only? If there it isn't, could this be considered an FR?

Thanks,

[adamread]

I've had this request from a number of non-publisher developers as well. This doesn't seem to be supported by AMP in any use case (client- or server-side).

As per my comment on 15653, being able to add or retrieve key-value pairs to a config object with amp-consent and then unblock components using a parameterized data-block-on-consent attribute would be a simple, flexible solution.

More granular consent seems to be a growing requirements in the EU. I've had a number of potential AMP implementations cancelled at legal review due to this.

[jawadst]

Wanted to chime in here and indicate that that's a use case that we see pretty often as well.

The overall use case is being able to granularly decide whether to load a tag or not based on consent to a purpose and/or a vendor and not on the global consent flag. On regular web, that logic is handled by the CMP and/or the tag manager.

Also note that part of the difficulty with AMP is that it only supports the IAB consent string or the global consent flag but not granular choices for non-AMP purposes or vendors which is pretty important to publishers, and especially to non-publishers.

[zhouyx]

I could see a potential solution is for AMP to come up with a framework that CMPs and publishers follow.

It's unlikely that AMP will understand different purposes, the object will likely only contain a list of key and decision pair. something like

{
  'globalKey': 'accept'
  'ad': 'reject',
  'analytics': 'accept'
}

Then the document will look like

<amp-fake data-block-on-consent/> will not be blocked
<amp-ad data-block-on-consent-ad/> will be blocked
<amp-ad data-block-on-consent-analytics/> will not be blocked

And then AMP will stringify the result to something like

{
  'v': 'v,1,ad,0,analytics,1'
  't': 1232543546547656 //timestamp
}

This could help to customize different section behavior, but it's still up to publisher to come up with such an object. Note that once stored, they won't be able to re-prompt to change the info. We would also need to consider the case when such an object and the consent string both exist.

[adamread]

This, unfortunately, wouldn't be a viable solution. Part of the rules involve the user being able to revoke their consent at any time. This also means that AMP needs a process to update vendors or block tracking and measurement when consent decisions are changed. Any solution that doesn't allow for this won't be usable by European developers.

[zhouyx]

The AMP team decided to only reapply consent decision changes the next time user visit the page for a few reasons.

1. For ad requests that had already been make, it's unclear if such request should be made again with consent decision changes.
2. For tracking requests, (if blocked by the data-block-on-consent) it's unclear if previous requests should be batched and sent if use change consent from reject to accept. Blocking future requests when the user change consent decision from accept to reject may be possible, but applying that decision to ready-to-be-sent request require major change to the current implementation. If it is blocked by the CONSENT_STATE macro, then analytics vendor can already get the updated consent state value, and handle request accordingly on server side.

Let me know what use case I'm missing. Could you please give me an example of how EU developers want to customize the behavior based on user consent decision, and how to update the page whenever the decision changes. Thanks.

[adamread]

Batched analytics ready for transmission are likely find to send, since the user had given consent when the measurement was made. The user can appeal to the site owner to have that data erased if they choose. The same is likely valid for Ads.

My understanding is that any new requests need to comply with the most recent consent decision, which must allow for granular consent. If the site owner is using an iframe and a CMP, then the data from the CMP either needs to be merged into the AMP consent state on update, or a URL to the CMP needs to be made available for the analytics provider to retrieve the IAB consent string. This also seems to lead to a requirement that analytics providers check the CMP URL every time before sending data, since the user could have updated their settings on another page.

[maranag]

I would also like to +1 this FR - this is indeed common and inability to implement this type of UI is a blocking issue for a fairly major investment in AMP.

[jawadst]

@zhouyx I also want to mention that the list of cookie categories listed are not legally defined. Websites will use different lists depending on what they are actually doing so AMP could try to standardize it but it will not work for all use cases.

Ideally, something more flexible where websites can define their own categories and match that through attributes (data-block-on-missing-consent="my-custom-category-name";

[micajuine-ho]

Hi all, we are looking to pick up this granular consent work soon. Have the requirements/use-cases for this feature changed or are we, in general, looking to block and unblock components based up on granular decisions made?

[hrkhal]

@micajuine-ho Are you planning to have this granular consent work completed before the TCFv2 deadline of 15th August?

Additionally, this granular consent should also allow us to block individual RTC vendor callouts for amp-ads in addition to the use cases specified in this issue.

[annasharkovamrf]

@micajuine-ho nice! 1) Would it be possible to use the data returned by the checkConsent endpoint as the config object? 2) Could you already describe a little the idea of implementation or is it too soon?:)

[micajuine-ho]

@hrkhal @annasharkovamrf

Are you planning to have this granular consent work completed before the TCFv2 deadline of 15th August?

We have no deadline yet as we are still scoping out the work. But probably will be later than Aug 15.

Additionally, this granular consent should also allow us to block individual RTC vendor callouts for amp-ads in addition to the use cases specified in this issue.

Noted, thanks! Edit: Currently, I believe we do not block RTC calls, we pass allow for the CONSENT_STATE macro to passed to the vendors, and then let them handle the information server-side.

Would it be possible to use the data returned by the checkConsent endpoint as the config object? Could you already describe a little the idea of implementation or is it too soon?:)

We are still figuring out requirements and coming up with a proposal. Will share here once it's ready.

[MedLeon]

Hi all, we are looking to pick up this granular consent work soon. Have the requirements/use-cases for this feature changed or are we, in general, looking to block and unblock components based up on granular decisions made?

Yes! I would estimate that 87% of the websites in the EU uses something similar. Like mentioned before the categories should not be predefined. Some Websites require two groups, some five.

PS: As far as I know this is the reason why AMP is not being implemented on many smaller websites in the EU. This is a key feature that is a deal breaker. How soon can we expect a solutions? I would start developing with the AMP Framework if I knew that this deal breaker will be goon soon.

[micajuine-ho]

Hi @MedLeon, the categories would be defined by the publisher/CMP then? Can you give some examples on the types of categories?

[MedLeon]

Hi @micajuine-ho ,

The publisher of the site should have control over the groups. He is the one in charge concerning the data protection.

Usually websites use 4 groups:

• necessary (=Cache-Cookies, Shop-System Cookies, Login-Cookies, Cookie Consent Settings Cookie)
• functional (= cookies that are helpful but not necessary for the main goal of the site; For example: wishlists)
• tracking and performance (=data used for analyzing )
• Targeting and ads (=Retargeting, Custom Audience)

The first group (necessary) is only displayed for clarity reasons and can't be unchecked. Thus the AMP Framework can ignore this group. Some sites doesn't even bother to display it at all and that is legally allowed. The other ones are by default unchecked and the user can decide wether or not he wants to check them. There are 2 buttons: "Save" and "Accept all".

It is mandatory that the user can decide which group they want to use and which not. So if a user doesn't want to be analyzed but likes to get useful ads, he may decide this way.

What the AMP Framework need to offer so small to medium sized businesses can use AMP:

• An easy way to define at least 3 groups.
• An Cookie Banner with...
  1. checkboxes for the groups (which are by default unchecked)
  2. a "Save" Button who saves the chosen groups
  3. an "Accept all" Button which saves all groups as accepted
• An easy way to unblock scripts/iframes/amp-analytics based on the different groups

Please keep in mind that most small business want an easy to implement script that doesn't require server requests at all.

If you have any questions - please let my know.

Kind regards :)

[jawadst]

@micajuine-ho Different publishers/CMPs have different categories defined. Ideally, AMP would support a customizable list that every CMP/publisher can define for their own operations. We would then need to be able to define blocking behaviors based on those categories.

Please also note that, from a GDPR perspective, consent should be based on both a set of purposes (categories here) and vendors. Publishers usually work with TCF vendors that define a set of purposes and vendors and encodes them in the consent string. Publishers also work with non-TCF vendors and define their own set of purposes and vendors to cover those non-TCF cases.

Ideally, a blocking behavior should be configurable on a combination of a vendor and a set of purposes/categories. That would allow us to cover the whole GDPR. For instance, we want to load Google Analytics if the user has given consent to the "Analytics" purpose and to the "Google" vendor. A user could deny consent to either Analytics or Google and Google Analytics should be blocked as a result. The granularity at the purpose and vendor level is pretty important to achieve from a legal perspective.

[morsssss]

/cc @SimonHogstromRonbol . Simon, note that I've already got your design and comments on this. Which we can also copy into this space.

Just noting a few things here:

• a set of checkboxes, where the user can select any options, would allow users to opt in or out of individual categories of cookies. If we also allow a set of radio buttons, where only one option could be selected, that would also enable a use case like the following:

In other words, instead of the user selecting A, B, or C - they could select A, A+B, or A+B+C.

• It's also essential (and required by law) that the user be able to expand each option to learn more about which cookies it would enable. This might not be a great case for an accordion, since this will happen in a popup of limited size. I've also seen cases where each expanded option allows for an even more granular list of cookies (corresponding to individual providers), but I don't know if that's a requirement for most people.

• I believe that the user should have the opportunity to change their consent choices each time they return to the site. I haven't actually seen this in action; does anyone (perhaps @adamread) know how this looks? I assume users aren't prompted to make cookie choices on every visit. (Or are they?)

• Finally, consent extends to collecting user data through forms.

[morsssss]

Also, a question for people who have commented here: did you find the documentation suitable? I'm wondering if I (or someone) should reorganize this, if time permits.

[SimonHogstromRonbol]

All of the suggestions here are really good, and we're really looking forward to having a consent solution that we make compliant with the GDPR rule set.

A lot of the suggestions here though, are centered around different groups, categories of cookies and such, and suggesting the amount of categories and such, while that is great as a snapshot of the situation right now, i think it is important to recognise that the GDPR and CCPR rule sets are not set in stone, and therefore, we really don't want to limit ourselves with a set number of categories and a certain way to setup the content inside the amp-consent modal, it's going to be impossible to keep up with the changes happening around the world in that regard.

Right now, we are limited by the fact that you can only have one amp-consent component on your site, and that amp-consent component can only have one status (accept, reject or pending).

A suggestion for an MVP would be for amp-consent, instead of only accepting one object of rules for consent, we would allow amp-consent to accept an array of objects, so that one amp-consent component can handle several levels of consent. something like this rough example:

<amp-consent id="mainConsent" layout="nodisplay">  
  <script type="application/json">  
    [{  
      "consentInstanceId": "consent1",  
      "consentRequired": true,  
      "promptUI": "consent"  
    },{  
      "consentInstanceId": "consent2",  
      "consentRequired": true,  
      "promptUI": "consent"  
    },{  
      "consentInstanceId": "consent3",  
      "consentRequired": true,  
      "promptUI": "consent"  
    }]  
  </script>  
  <div id="consent">  
      <h2>Consent</h2>  
      <button on="tap:consent1.reject,consent2.reject,consent3.reject">Reject all cookies</button>  
      <button on="tap:consent1.accept,consent2.accept,consent3.reject">Accept some cookies</button>  
      <button on="tap:consent1.accept,consent2.accept,consent3.accept">Accept all cookies</button>  
  </div>  
</amp-consent> 

Then we let the developers actually control the content of the amp-consent modal using amp-list and amp-bind, and if we really want to help developers, we can update the documentation on amp.dev with examples of how you could set up checkboxes, lists of cookies and more.

@morsssss at Online-Results we'd start working on a GDPR compliant modal for amp-consent as soon as this update has launched, so we'd be able to put that in the amp.dev documentation, as an example, right away.

[morsssss]

Agreed on all points. GDPR and related privacy rules will keep on evolving, so this solution should allow maximum flexibility while preserving AMP's performance guarantees.

• An array of objects. Over here we've been talking about the same idea - letting developers define the categories of consent.

• A form-like user consent experience. Then, the developer can create a form-like DOM element (perhaps even a form) in which those categories can be inputs. I'm proposing that these inputs would normally be checkboxes or radio buttons. Would there ever be a use case for text inputs? (The input would need to be sanitized in such cases, of course.)

• Expandability. The user should also be able to expand categories to either get more information or to make individual privacy choices within a category. I feel like this can be done with <amp-bind> and <amp-accordion>. You mentioned once that you would make use of <amp-list> here. Really, we should simply allow use of the usual AMP components.

• Use. The developer could then use a consent category - say, "marketing" - in an attribute like data-block-on-marketing.

If the user needs to have the chance to return to the site and change their consent choices, how should that option be presented?

Let me know if we're missing other features...

[chris-hinds]

All of the suggestions here are really good, and we're really looking forward to having a consent solution that we make compliant with the GDPR rule set.

A lot of the suggestions here though, are centered around different groups, categories of cookies and such, and suggesting the amount of categories and such, while that is great as a snapshot of the situation right now, i think it is important to recognise that the GDPR and CCPR rule sets are not set in stone, and therefore, we really don't want to limit ourselves with a set number of categories and a certain way to setup the content inside the amp-consent modal, it's going to be impossible to keep up with the changes happening around the world in that regard.

Right now, we are limited by the fact that you can only have one amp-consent component on your site, and that amp-consent component can only have one status (accept, reject or pending).

A suggestion for an MVP would be for amp-consent, instead of only accepting one object of rules for consent, we would allow amp-consent to accept an array of objects, so that one amp-consent component can handle several levels of consent. something like this rough example:

<amp-consent id="mainConsent" layout="nodisplay">  
  <script type="application/json">  
    [{  
      "consentInstanceId": "consent1",  
      "consentRequired": true,  
      "promptUI": "consent"  
    },{  
      "consentInstanceId": "consent2",  
      "consentRequired": true,  
      "promptUI": "consent"  
    },{  
      "consentInstanceId": "consent3",  
      "consentRequired": true,  
      "promptUI": "consent"  
    }]  
  </script>  
  <div id="consent">  
      <h2>Consent</h2>  
      <button on="tap:consent1.reject,consent2.reject,consent3.reject">Reject all cookies</button>  
      <button on="tap:consent1.accept,consent2.accept,consent3.reject">Accept some cookies</button>  
      <button on="tap:consent1.accept,consent2.accept,consent3.accept">Accept all cookies</button>  
  </div>  
</amp-consent> 

Then we let the developers actually control the content of the amp-consent modal using amp-list and amp-bind, and if we really want to help developers, we can update the documentation on amp.dev with examples of how you could set up checkboxes, lists of cookies and more.

@morsssss at Online-Results we'd start working on a GDPR compliant modal for amp-consent as soon as this update has launched, so we'd be able to put that in the amp.dev documentation, as an example, right away.

This sounds like exactly what we need here at the BBC. We currently have this level of granular consent on our canonical pages and we need the same style of implementation on our AMP pages.

Without this level of granular consent we are very limited in what we can do on these AMP pages and we want todo more i.e. collect core web-vitals, serve personalised ads (outside of the UK) etc.

[morsssss]

Thanks for joining the discussion, @chris-hinds ! The contributors have been gathering requirements here for a little while, and we need all the input we can get from developers like you.

The intention is definitely to provide an API that lets developers create their own categories of consent, create a UI where users choose what to block and to allow, and block/allow individual components or classes of components accordingly.

Actually, here are a few questions for you, for @SimonHogstromRonbol , for and anyone else who has an opinion.

fallbackAction. You already know that amp-consent lets you specify a fallbackAction option - either reject or dismiss. This is used when the user doesn't make a consent choice within a certain amount of time (the timeout value). The question is - would you ever need to have different fallbacks for different consent categories or cookies?

In other words, would there ever be a case where, if the user didn't make a consent choice, you'd want to block one component but not another?

It would be simpler overall to make the fallback the same for every consent category on a page. But please let us know if you would need to customize it.

timeout. Similarly, it would be simpler to make the timeout the same for every consent category. But let us know if you'd ever want to make this, say, 5 seconds for one category and 10 seconds for another.

third-party consent managers. Most websites I've seen use a third-party solution to manage consent. Of course, these won't work out of the box with amp-consent's new granular consent until the third parties have had time and a business need to implement AMP-compatible solutions. Do you feel comfortable implementing this yourself on AMP sites, without assistance from a third-party solution?

[chris-hinds]

Thanks for getting back to me @morsssss

fallbackAction. You already know that amp-consent lets you specify a fallbackAction option - either reject or dismiss. This is used when the user doesn't make a consent choice within a certain amount of time (the timeout value). The question is - would you ever need to have different fallbacks for different consent categories or cookies?

Speaking with my team I think we'd be happy with a default fallback for all consent values. And for us that would probably default to rejecting consent (This may change in the future and obviously may be different for others) although perhaps the ability to configure the fallback would be good (if that cant be done already)

timeout. Similarly, it would be simpler to make the timeout the same for every consent category. But let us know if you'd ever want to make this, say, 5 seconds for one category and 10 seconds for another.

I cant yet see a use case for this for us. I get trying to make this as simple as possible but perhaps there just needs to be default values for each consent value which can be overridden if needed as requirements can and will always change.

third-party consent managers. Most websites I've seen use a third-party solution to manage consent. Of course, these won't work out of the box with amp-consent's new granular consent until the third parties have had time and a business need to implement AMP-compatible solutions. Do you feel comfortable implementing this yourself on AMP sites, without assistance from a third-party solution?

This isn't actually something I've looked into very much. So far we have relied on the amp-consent component only. We had looked into using web-packaging and signed exchanges to allow access to 1st party cookies however last I checked we were still waiting on our CDN provider to support this at their level.

[SimonHogstromRonbol]

I agree with a lot of what @chris-hinds is saying

FallbackAction and Timeout For us, having different fallback options for different categories won't be necessary. I can’t, off the top of my head, come up with a use case for having different fallbacks for different categories, since everything is being handled from the same amp-consent modal, same goes for timeout.

Third-party consent managers We’ve been working with different providers. What those services excel at are a really good plug-n-play solution for complex sites to make them GDPR compliant, without any thought given to what cookies are on those sites, this really isn’t what we are looking for with AMP.

One of the advantages with AMP as our framework of choice, is that we know about every cookie being set on the site, and we would be able to handle them individually.

One thing to add The only thing I feel would be really helpful for developers in this case, would be to add cookie information to each of the AMP components on the amp.dev documentation. For instance, if you are a developer and you want to add a youtube video to your site, it would be super helpful to have a list of the cookies being set on your site by using the amp-youtube component, as well as information about those cookies, since, according to the GDPR you are required to provide this information to your users - so every individual AMP developer would have to figure out all of this information on their own, if they want to follow the law.

[micajuine-ho]

Hi all! Thanks for the continued discussion and shout out to @morsssss for getting the conversation going again!

I have written up a doc here with a proposal for introducing client side granular consent into amp-consent. I would love to get feedback regarding the design and if it satisfies your use cases.

Thanks

/cc @zhouyx

[SimonHogstromRonbol]

@micajuine-ho That doc looks very comprehensive, and looks to be a really good summary of everything discussed.

Something we're missing for a usecase that comes up a lot when dealing with the GDPR is collection of data outside of tracking, for instance, using forms, when a user rejects tracking of personal data, they shouldn't be able to fill in a form on the page, but right now, even if you data-block-on-consent on the form element itself, you are still able to send messages using that form.

I'd propose an attribute added to the element that has data-block-on-consent or data-block-on-consent-purposes on whether consent has been accepted or rejected, to let us respond to a consent rejection using CSS.

data-consent-rejected and data-consent-accepted, of, if you prefer, something like data-consent-status="rejected".

With an attribute like that we'd be able to inform the user why the google maps iframe is just showing a spinner and never loading, we'd be able to disable a button in my form, or add a TOS checkbox to the form, if the user who've rejected tracking, until consent is updated.

_edit: An example of a bad user experience would be this example given on amp.dev where if consent is rejected, the spinner for the rejected image just keeps spinning and never loading. An attribute on the amp-img element here, would make us able to react, and make a better user experience._

[micajuine-ho]

@SimonHogstromRonbol Thank you for your feedback.

Something we're missing for a usecase that comes up a lot when dealing with the GDPR is collection of data outside of tracking, for instance, using forms, when a user rejects tracking of personal data, they shouldn't be able to fill in a form on the page, but right now, even if you data-block-on-consent on the form element itself, you are still able to send messages using that form.

For <forms> (as well as <div>s and <p>s), we recommend wrapping them in an <amp-layout> tag and then adding the data-block-on-consent (and future data-block-on-consent-purposes) attribute on it so that the form will be blocked when consent is rejected.

I'd propose an attribute added to the element that has data-block-on-consent or data-block-on-consent-purposes on whether consent has been accepted or rejected, to let us respond to a consent rejection using CSS.

data-consent-rejected and data-consent-accepted, of, if you prefer, something like data-consent-status="rejected".

With an attribute like that we'd be able to inform the user why the google maps iframe is just showing a spinner and never loading, we'd be able to disable a button in my form, or add a TOS checkbox to the form, if the user who've rejected tracking, until consent is updated.

edit: An example of a bad user experience would be this example given on amp.dev where if consent is rejected, the spinner for the rejected image just keeps spinning and never loading. An attribute on the amp-img element here, would make us able to react, and make a better user experience.

I definitely see the issue here and I actually think that it warrants it's own feature request--as I wouldn't want to squish this into the granular consent feature. I think it's something we can tackle after this project.

[morsssss]

To folks on this thread - if you weren't aware, you can track work on this in https://github.com/ampproject/amphtml/issues/31607. It's been available behind an experiment flag since earlier this month. We'd like you to try this out and tell us if you notice anything confusing or surprising!

I've created a sample which contains a more detailed how-to here. You can look at the code or clone it here.

[morsssss]

@chris-hinds , @jawadst , @annasharkovamrf , @jawadst , @hrkhal - if any of you would like to try this out, please do. Please check the sample for guidance. And let us know if you spot anything...