I2I: Allow receiving TCF 2.0 consent string via TCF postmessage API

klipstein commented 4 years ago

Summary

Currently only <amp-ad> and <amp-iframe> have the capability to receive the TC-String, which could be gathered via <amp-consent>. We would like to enable more components to receive the TC-String by implementing a proxy between <amp-consent> and the TCF 2.0 postmessage API.

This idea was already mentioned by @renebaudisch in this thread and we would like to propose and discuss a potential solution that implements a minimal TCF 2.0 API.

All consumers that rely on TCF 2.0 already have TCF 2.0 reading implemented and this feature would prevent that those consumers need to adjust their code.

Other use-cases where the consent-string cannot be queried at the moment:

When an ad gets delivered via <amp-ad> and where the ad is delivered via safeframe
Standardize receiving the consent-string for prebid integrations: https://github.com/prebid/prebid-universal-creative/issues/114

Design Document

We would like to extend <amp-consent> so that a publisher can expose his AMP CMP via the TCF postmessage API as such:

Option A

<amp-consent
  exposes="tcfapi"
  ...
></amp-consent>

Option B

<amp-consent ...>
  <script type="application/json">
    {
      // ...
    exposes: ['tcfapi']
    }
  </script>
</amp-consent>

With assigning multiple values to exposes, it would be possible to extend this feature to support further consent APIs (e.g. uspapi). Alternative names to exposes would be installs, or provides.

When a publisher exposes his CMP as tcfapi the following happens:

Installs a new window frame with the name __tcfapiLocator next to the top window, which indicates to a foreign iframe that a TCF 2.0 API is available. A foreign iframe can query for the existence of a TCF API by testing for window.top.frames.__tcfapiLocator.
Within the top window we then would install listeners for the getTCData, addEventListener removeEventListener and ping window messages as specced here and here
When a TCF API request is received in one of those listeners, call the AMP consent APIs in order to gather consent information, which were stored within AMP (The used AMP APIs will be: getConsentMetadata, getConsentPolicyState, getConsentPolicyInfo and getConsentPolicySharedData) and use that stored data to answer with a minimal TCData response.

Because not all necessary data of a TCF 2.0 compliant CMP is stored within AMP, it is not possible to construct a completely filled TCData object. The most important information within the TCData object is the tcString and a lot of information within the TCData object is just expanded information that is encoded within the tcString. Because of that we propose a minimal TCData object that looks like this:

const metadata = await getConsentMetadata();
const policyState = await getConsentPolicyState();
const policyInfo = await getConsentPolicyInfo();
const sharedData = await getConsentPolicySharedData();

const tcData = {
  // mixing in additionalConsent and sharedData from a CMP
  additionalConsent: metadata.additionalConsent,
  ...sharedData,
  consentStringType: metadata.consentStringType, // TCF-Version
  gdprApplies: metadata.gdprApplies,
  tcString: policyInfo,
  // will be defined when "addEventListener" is used
  listenerId: undefined,
  // this can be defaultet to "loaded" when e.g. getConsentMetadata answered 
  cmpStatus: 'loaded',
  // same as above, when we receive an answer, we can assume that everything is ready
  eventStatus: 'tcloaded'
};

Motivation

We've implemented the consent-reading support for amp-iframe and we now wanted to provide the same functionality for <amp-video-iframe>. Because other components (e.g. other video player components) will also have the requirement to follow the TCF 2.0 standard we want to offer a more generic way to distribute it and we think that the TCF 2.0 postmessage specification solves this problem already.

Alternative solution would be, that all AMP components, that require TCF 2.0 consent reading would have to implement it.

/cc @ampproject/wg-approvers /cc @zhouyx, @jridgewell, @alanorozco (because you reviewed https://github.com/ampproject/amphtml/pull/29636)

klipstein commented 3 years ago

@zhouyx, @jridgewell, @alanorozco what can we do to move this forward?

jridgewell commented 3 years ago

It needs to be brought do a design review, with @ampproject/wg-monetization in attendance. /cc @zhouyx

micajuine-ho commented 3 years ago

Hi @klipstein, thank you for your proposal. I have written up a design doc for this work, and I would love some feedback from the community before I take it to design review.

Thanks!

klipstein commented 3 years ago

@micajuine-ho thanks for writing the detailed design doc. Maybe you could express better that the postmessage listeners need to be installed on the frame that contains the __tcfapiLocator frame (which mostly will be the top window). But other than that it looks good to me.

micajuine-ho commented 3 years ago

I'll track the work done for this project here:

[x] Add exposes API to amp-consent (#31235)
[x] Install the __tcfApiLocator window (#31235)
[x] Allow PostMessages for the ping, getTcData, addEventListener, and removeEventListener commands (#31235)
[x] Create the ping API (along with the MinimalPing object) (#31798)
[x] Create the getTcData API (along with the MinimalTCData object) (#32411)
[x] Create the addEventListener API (#32552)
[x] Create the removeEventListener API (#32552)
[ ] e2e tests / Documentation

/cc @zhouyx

renebaudisch commented 3 years ago

Is there anything the crowd can help to push this forward?

micajuine-ho commented 3 years ago

Got positive feedback from the design review today, will look to move forward with these PRs

micajuine-ho commented 3 years ago

@klipstein & @renebaudisch PRs have been merged under an experiment flag. Please let me know when you get a chance to test this feature, so we can continue with the rollout

klipstein commented 3 years ago

@micajuine-ho just tested the experiment in nightly, beta and experimental channel and it works for me. Getting the adjusted consent-string via addEventListener when the consent was adjusted is great. Also tested it through some 3rd party components and all of them use addEventListener to receive the latest consent-string.

When will that experiment be available in the production build?

renebaudisch commented 3 years ago

I also tested that feature and it seems to work well, even it really is a very small set of information and I was hoping to also get granular input to e.g. the purposes as well. Would be great if the info passed could be extended, but the Implementation of the postMessage proxy works as expected.

Thanks for the effort.

micajuine-ho commented 3 years ago

@renebaudisch @klipstein Thanks for the feedback, I will be turning on the experiment, which should be available in production in 2 weeks.

@renebaudisch What other information would you like passed? This can be follow up work.

renebaudisch commented 3 years ago

It would be great if amp would already analyses the string at least for purpose 1 to 10 if not also for the vendor consents as they are officially listed and therefore can be interpreted from the string, but that's sure a nice to have.

micajuine-ho commented 3 years ago

AMP consent offers explicit purpose 1 support: https://github.com/ampproject/amphtml/pull/31145

We can look into the other purpose support as well.

klipstein commented 3 years ago

@micajuine-ho we successfully tested reading the consent string when amp-video-iframe is used. What are the next steps so that this feature gets released officially (including documentation)?

micajuine-ho commented 3 years ago

Experiment is fully launched, need to do documentation and clean up now. Glad it was successful!

skibi123 commented 3 years ago

I tried new features and I have one question: There is a note in TCF 2.0 docs:

"Note: The addEventListener callback shall be immediately called upon registration with the current TC data, even if the CMP status is loading and the CMP has incomplete TC Data, so that the calling script may have access to its registered listenerId. Furthermore, on every TC String change the callback shall be called unless it is removed via removeEventListener."

And I observed that this immediate call of callback is not performed. Is it an error ?

klipstein commented 3 years ago

@skibi123 as far as I understand the AMP consent implementation: there is no "loading" or "incomplete TC Data" state and therefore it is not possible to call addEventListener. I would recommend to block the third party component with data-block-on-consent="_till_responded" so that your code will only be loaded when a user interacted with the CMP.

klipstein commented 3 years ago

@micajuine-ho when will the referenced pull request #34954 be merged?

renebaudisch commented 3 years ago

@skibi123 there is a state cmpUiShown thats signals that the user is prompted for input. When you do your addEventListener registration meanwhile, the callback you defined will be triggered immediately, even the user still needs to do his choices. It will also be called on each following event, like "userActionComplete" unless you remove your Listener.

This means, your callback defined for addEventListener shouldn't be a one shot since you never know what state the user is in, but you can check it inside the callback by "tcData.eventStatus".

Consent information is ready on "tcloaded" or "useractioncomplete".

renebaudisch commented 3 years ago

@micajuine-ho There isn't any documentation on this feature and that "exposesTcfApi" even exists. This way its hard to make our partners aware of it and how they shall use it. When can we expect a documentation?

kristoferbaxter commented 3 years ago

@sebastianbenz and @alanorozco can your teams help assist with documentation here?

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

ampproject / amphtml