Open zhouyx opened 4 years ago
@AmazonPublisherServices @leonardlabat @bretg @wroberts-comscore for feedback since you have dealt with CONSENT_STATE and CONSENT_STRING in AMP.
@yoannOgury @PierigOgury @andresilveirah @jawadst @tla-sirdata for feedback from CMP
Thank you!
Prebid Server receives RTC calls from AMP. We find it sub-optimal to have to introspect CONSENT_STRING, which could be formatted as a TCF1.0 or CCPA string.
Would it be possible to add a "CONSENT_TYPE" attribute which the CMP can add to provide a clear signal on how the string is formatted? Valid CONSENT_TYPE values could be: TCF1, TCF2, USP, etc.
How difficult it is for ad vendors to interpret consent string without the TCF APIs
Speaking for Prebid, if we had CONSENT_TYPE, we'd be happier parsing strings. We do it already in Prebid Server. In fact, we don't currently use the AMP CONSENT_STATE -- we assume that the user consent state is already encoded in the string.
It's unsatisfying and won't scale well to parse the consent string with a set of rules like:
Thank you @bretg for the feedback. TCF1.0 and 2.0 should be easy to interpret as the version number is included in the TC string. But I agree that the US privacy string make things more difficult. Is there a standard name that the AMP team can follow? Could you please let us know how CMPs are passing this information in non AMP page today? Thank you
In non-AMP pages, there are different IAB functions in the page that serve as the interface. For GDPR, the ad tech javascript infrastructure calls window.cmp(). For CCPA it's window.uspapi(). So because these separate interfaces exist, it's quite clear to the ad ecosystem how to parse the results of the call.
Being on the vendor end of RTC is quite different - we're not making the call to the CMP ourselves so don't have the context.
FWIW - prebid.js requires that publishers turn on GDPR and USPrivacy and let it know which specific interfaces to use. e.g.
pbjs.setConfig({
consentManagement: {
gdpr: {
cmpApi: 'iab', // could also be passed in statically
...
},
usp: {
cmpApi: 'iab', // could also be passed in statically
...
}
}
});
As a CMP first option seems attractive, it will give us more flexibility, each vendors will be able to use the exposed api or the consent string, without any specific intégration for AMP, it is the idea behind tcf implementation. This option also allows the most limited liability on AMP side.
On AMP side, the second option will need less engineering and is clearly more safe technically, and dealing with CONSENT_TYPE seems a good simple solution .
However it will force vendors to find a solution to be able to decode consent string, with one decoder by consent string type, it could be really painful for vendors, with some performances impact.
Thats why as a CMP, we prefer the first option, but the second one works also with more impacts for vendors.
In all cases, we think that AMP have to keep the hand on which kind of consent string can be provided through the amp-consent.
At the same level as the CONSENT_TYPE remark, it would also be useful to be able to set multiple consent signals (in case of a CMP managing multiple framework/regulation with one notice to the end user)
dealing with CONSENT_TYPE seems a good simple solution. However it will force vendors to find a solution to be able to decode consent string, with one decoder by consent string type
Not quite sure I understand you here @PierigOgury - are you suggesting that the existence of CONSENT_TYPE makes parsing harder? My take is that each RTC vendor has to have code to parse all encodings anyhow... having metadata about the encoding format just means no guessing.
Hi @bretg , I mean that the first solution proposed allows to not have a decoder in case the API gives an "easier" to manage information (such as the TCF API regardless of the version). With CCPA framework it is mandatory to have a "decoder" as the API is sending only the encoded part. With the second solution a decoder is mandatory regardless of the API defined by the framework, for anyone in the amp environment (vendor and publisher). And as you mention, the CONSENT_TYPE allows for no guessing on the type of object received.
Thanks all for the feedback.
After more discussion, the AMP team decided Option #1 is not very practical. As @PierigOgury pointed out it requires a lot engineering resources. Also it's not technically safe because AMP is not a registered CMP. Because of this we decided to stay with the current approach (Option #2)
Re: CONSENT_TYPE
. Sounds reasonable to add the additional information.
I only have one concern, the CONSENT_TYPE
only makes sense when it is stored with consent string together and can be understand by all parties. @PierigOgury @bretg does this mean that AMP needs to declare the types of consent string framework we support and provide a list of standard type names?
Hi, First sorry for my late feedback. I would like to bring more elements even if it seems the AMP team has made its decision.
I think the fact CMPs have no control on whether to display the UI is a problem. When the vendor list changes, we need to display the UI to ask consent for additional vendors while keeping previous choices. With the actual integration, we can only expire the consent string, which is obviously not what we want. TCF v2.0 also introduces a TCF Policy version that we have to take in consideration to display the UI when it changes. Additionally, the recent CNIL notice (still in public comment) indicates that the UI should be displayed again in case of "significant change". These are just some examples but we can see there can be many reasons why CMPs need to display the UI, and they change over time depending regulations and frameworks. If AMP keeps control on the display, it must ensures those rules will be applied, which I think is painful for AMP engineers and doesn't make sense as AMP is not a CMP.
As a reasonable solution, CMPs could add an attribute in the check endpoint's response to indicate to AMP that we want to display the UI while keeping the existing consent string. On AMP side, a capping could be managed to prevent CMPs from displaying the UI too often. This solution doesn't require a CORS iframe.
About reading consent data, it would be indeed very convenient for vendors to call the standard APIs and not have to decode consent strings. Actually in TCF, vendors are not supposed to decode the consent string by themselves, they should always read consents through the CMP API, which ensures them that signals are valid and the consent string is not corrupted. That point should probably be discussed with the IAB Europe as they probably didn't design the framework with AMP concerns in mind.
The limitation I see in the actual integration is that CMPs can only transmit a consent string (and maybe a future consent type). As mentioned in the discussion's description, publishers may need to work with vendors outside the GVL list (example: Google before TCF v2 adoption), so we have to transmit those additional information.
The main idea is to provide CMPs a way to transmit more information (at page load and after user submit), the challenge being to give enough flexibility to CMPs while keeping a standardized AMP structure. As a workaround to standard APIs, CMPs could then use that feature to expose decoded data to vendors. Once again no CORS iframe is needed.
Talked to @jeffjose offline. We agreed that AMP can provide such CONSENT_TYPE
information. TCF v1, v2 and US privacy string will be the three formats we support for now. In the future we are open to add other format to the list upon requests.
The team is currently focus on adding us-ca detection capability to amp-geo, and doesn't have bandwidth to this API this sprint. We'll update the thread once we pick up the work. Thanks.
As a reasonable solution, CMPs could add an attribute in the check endpoint's response to indicate to AMP that we want to display the UI while keeping the existing consent string. On AMP side, a capping could be managed to prevent CMPs from displaying the UI too often. This solution doesn't require a CORS iframe.
The only concern is user experience. @micajuine-ho @jeffjose and @lannka would like to get your thoughts on the tradeoff here.
The main idea is to provide CMPs a way to transmit more information (at page load and after user submit), the challenge being to give enough flexibility to CMPs while keeping a standardized AMP structure. As a workaround to standard APIs, CMPs could then use that feature to expose decoded data to vendors. Once again no CORS iframe is needed.
We could provide an API for CMP to pass consent data object to vendors. AMP won't store the data object or enforce the object format. My concern is that without standard structure, such type of object cannot be parsed by vendors.
The only concern is user experience. @micajuine-ho @jeffjose and @lannka would like to get your thoughts on the tradeoff here.
Sorry @zhouyx, are you asking me to give my thoughts about the tradeoff here ?
@zhouyx wanted to see where the AMP support for TCF version 2 stands. i'm having trouble understanding it's progress from this thread.
@zhouyx Having worked with many vendors in the last months on TCFv1 integration with AMP, the main piece of feedback we have is that most vendors are still unclear on how they can get access to the consent string from their AMP tags.
In particular, they are struggling with two elements:
We have tried to summarize all common cases on https://developers.didomi.io/cmp/amp/consent-status-for-vendors and https://developers.didomi.io/cmp/amp/blocking-behaviors but I'd recommend improving the AMP docs for vendors on that front for the V2. Happy to help if there is a way for us to contribute to the docs.
Thank you @jawadst This is very helpful! I've added you as reviewer to #27855
@cmaurersp any specific detail you need?
AMP attempts to support TCF v2 as a generic TC framework as best as we can. AMP will collect and store raw consent string from CMP and pass the raw string value to vendors. As described above AMP decided not to support the TCF v2 APIs though, and expect vendors to parse the string on the server side.
@cmaurersp any specific detail you need?
AMP attempts to support TCF v2 as a generic TC framework as best as we can. AMP will collect and store raw consent string from CMP and pass the raw string value to vendors. As described above AMP decided not to support the TCF v2 APIs though, and expect vendors to parse the string on the server side.
What do 3rd party CMPs need to do in order to support TCF v2 at this point in time?
If you want to integrate your CMP solution with AMP. For example
<amp-consent type='myCMP'></amp-consent>
Please follow the CMP integration guideline here. https://github.com/ampproject/amphtml/blob/master/extensions/amp-consent/integrating-consent.md Let me know if there's any question.
We are creating a CMP for our publishers and would like to do an integration test locally before we actually create a pull request. How can we create a full working demo?
Our demo looks like the following so far:
<amp-consent id="myUserConsent" layout="nodisplay">
<script type="application/json">{
"consentInstanceId": "consent-id",
"consentRequired": true,
"promptUIaaa": "consentDialog",
"postPromptUI": "post-consent-ui",
"checkConsentHref": "https://open-cmp-local-api.de:5003/consent/check",
"onUpdateHref": "https://open-cmp-local-api.de:5003/consent/update",
"promptUISrc": "https://open-cmp-local-iframe.de:5003/demo/amp/cmp-frame.html",
"storageKey": "cmp_foo_bar"
}</script>
<div id="post-consent-ui">
<button on="tap:myUserConsent.prompt()">Update Consent</button>
</div>
</amp-consent>
In Dev tools I can see a iframe being created loading promptUISrc, which is the standalone CMP. No errors reported. But the iframe is hidden and only a spinner is displayed. It's probably something I missed, could someone give me a hint?
Hi we would like to integrate our TCF v2 CMP to AMP and have some questions :
CONSENT_TYPE
field ?sirdata-v2
) ? Same question for CCPAThanks
Hi @saraneus. Thanks for your question. Your current implementation of adding your CMP to AMP seems a little off. To integrate into AMP as a CMP, you need to follow the CMP integration guideline here. https://github.com/ampproject/amphtml/blob/master/extensions/amp-consent/integrating-consent.md
After following these steps, you can test locally with a consent element similar to this (replacing _ping_
with your CMP name):
<amp-consent id='ABC' layout='nodisplay' type='_ping_'>
<script type="application/json">
{
"postPromptUI": "postPromptUI",
"clientConfig": {
"CMP_id": "test_id",
"other_info": "test_info"
}
}
</script>
<div id="postPromptUI">
Post Prompt UI
<button on="tap:ABC.prompt(consent=_ping_)" role="button">Manage</button>
</div>
Another hint is that
"checkConsentHref": "https://open-cmp-local-api.de:5003/consent/check",
"onUpdateHref": "https://open-cmp-local-api.de:5003/consent/update",
"promptUISrc": "https://open-cmp-local-iframe.de:5003/demo/amp/cmp-frame.html",
should be here under your new CMP name.
Please let me know if you have any further questions.
Hi @tla-sirdata, thank you for your questions.
Can you provide information about integration and the new CONSENT_TYPE field ?
This is still in the works. Please refer to #27907, and we can continue the discussion there.
How are we supposed to add the configuration here ?
To integrate into AMP as a CMP, you need to follow the CMP integration guideline here. https://github.com/ampproject/amphtml/blob/master/extensions/amp-consent/integrating-consent.md If you have any questions on these steps, please let me know.
Is there something to indicate v2 endpoints or should we create another config type (for example : sirdata-v2) ? Same question for CCPA
I believe you are asking if there is someway to tell if the consent_string
you are passing along through your endpoint satisfies TCF_V1, TCF_V2, or US Privacy String. If this is the case, then I believe that this issue can be solved with the addition of the CONSENT_TYPE field as metadata for the consent string (again, see the issue linked above).
TCF v2 requires a very large amount of information on the 1st layer, which can result to a dense content with the 60vh restriction. Can we consider increase it ?
@zhouyx correct me if I'm wrong, but I believe we decided on 60vh restriction due to the bad UX that would be created if users were to just see a fullscreen consent dialog. However, you can use our fullscreen API to enlarge the consent dialog.
@micajuine-ho Thanks for your answers !
I believe you are asking if there is someway to tell if the consent_string you are passing along through your endpoint satisfies TCF_V1, TCF_V2, or US Privacy String. If this is the case, then I believe that this issue can be solved with the addition of the CONSENT_TYPE field as metadata for the consent string (again, see the issue linked above).
I'm talking about the CMP version a publisher chooses to use.
Actually we're already integrated in AMP as a TCF v1 CMP and would like to add our TCF v2 version. I'm wondering if publishers will have to change the <amp-consent>
type (from sirdata
to sirdata-v2
) or if they will be able to switch with a parameter in the JSON config.
Thanks
@tla-sirdata Thanks for the clarification.
As a CMP do you anticipate that having publishers using v1 and v2 will be a common use case for you?
Something that we could do would be to include a CONSENT_TYPE field within the request to the CMP (in the checkConsentHref endpoint), that could be overriden by the inline publisher, but would otherwise default to the last CONSENT_TYPE stored in local storage.
I don't think we currently have a clear cut solution for this. @zhouyx for ideas/confirmation.
I have similar questions.
Also I don't know how AMP will deal with CCPA but we were thinking about detect geolocation through our v2 check endpoint to provide the good UI (CCPA or TCF v2). Does it make sense to you ?
amp-geo
provides the geolocation support via preset-us-ca
and preset-eea
We've developed two new endpoints (check + UI) for the TCF v2 so both versions could live in parallel. We could use the same endpoints to serve both versions but in this case we'll need to update our current endpoints.
It will work both ways. If you can reuse the same endpoint, then you can ask publishers to choose the TCF via the clientConfig
. If you want to have two endpoints, then you can integrate sirdata-v2
with us as a new CMP. But since the TCF v1 support will be deprecated soon, please let us know your deprecation plan with the existing sirdata
integration. Thank.
amp-geo provides the geolocation support via preset-us-ca and preset-eea
OK so it will be up to the publisher to implement geolocation and choose CCPA or GDPR.
It will work both ways. If you can reuse the same endpoint, then you can ask publishers to choose the TCF via the clientConfig. If you want to have two endpoints, then you can integrate sirdata-v2 with us as a new CMP. But since the TCF v1 support will be deprecated soon, please let us know your deprecation plan with the existing sirdata integration. Thank.
I prefer not to multiply CMPs. Passing the version via the clientConfig
sounds good to me as both endpoints receive it. We'll dig on this.
Thank you!
Hi I'm testing a TCF v2 integration and have the following error when I give consent :
log.js:251 [CONSENT-STATE-MANAGER] Cannot store consentString which length exceeds 200. Previous stored consentInfo will be cleared
Here is the consent string sent to AMP : COykPRuOykPRuBcABBENAjCsAP_AAH_AAAqIF5EX2S5OI2tjI2YdF5BEYYwfJxyigMgChgQIsS4NwIeFbBoGL2AAHBG4JCQAGBAEEACBAQIkHGBcCQAAgIgBiRCMQEmMCzNKBJJAggEbM0FACCVmHkHSmZCY7064O__TAvIgkwVJwCRIUAgBIsQIhDABAOIAABgCUEAAECFhoACAjYFAR6gAAAIDAACAAAAEEAAAIABAAAgIgAAAEBACACIBAAAAEaAhAABIEAEAJEgAAAQA0IACCIIQBCDgQCjhAAA
As you can check here, the TC String is valid for TCF V2 so the consent string length should be allowed to exceed 200 in AMP.
Thanks
Thanks for raising the issue. Yes, there's plan to expand the storage limit. We are still determining the desired storage space. More context on the AMP localstorage policy can be found here.
Hi all! Currently, consent dialogs can be configured to a max of 60vh, and be scrollable/expandable.
I wanted to get your opinions on if this current setup is sufficient to display all disclosures required by TCFv2, as well as the accept/reject buttons.
/cc @yoannOgury @PierigOgury @andresilveirah @jawadst @tla-sirdata for feedback from CMP
Hi, if we want to keep the UI as readable as possible for the user, 60vh is definetly not sufficient considering the amount of information CMPs have to disclose with the TCF v2. In my opinion making it scrollable is bad for user experience so we're not considering this option.
Another thing is that it might be useful to allow CMPs to exit fullscreen mode so the initial layer would always be displayed the same way (after a click on a Back button for example).
Thanks
We have some ideas to improve the UI for the consent dialog that we are taking to design review #28108 tomorrow. Please feel free to join or once we get feedback from the whole amp team, we can discuss further.
Hi @micajuine-ho do you have any update on the UI ? Thanks
Hi @tla-sirdata, we took our design proposals to design review last week, meeting notes can be found here.
We decided that option three (Immediate Lightbox iframe) is how we would like to proceed. Next steps will be to design an API for this lightbox consent. Please let me know if you have any feedback, questions, or concerns.
HI @micajuine-ho thank you, option 3 looks great to me.
I have some questions :
amp-consent
doesn't fit well (especially because of the 60vh fixed height). What do you think about it ? Is this a common case ? If so, we should consider a responsive design for amp-consent
or allow CMPs to access the window width to adapt their design.Thanks !
@tla-sirdata Thanks for the questions!
do this option will replace the current one or both will be available ?
We will design for both to be available, so that it is backwards compatible with older consent frameworks.
have you thought about my request to allow CMPs to quit the fullscreen mode after being triggered ?
For the immediate lightbox design, we are thinking to not allow the fullscreen API; in the common mobile case; the disclosure will take up as much space as possible (only limited by our lightbox margin), and so going from ~90% of the screen to 100% wouldn't really do much for the user.
Once the lightbox flow has been implemented, would you see a need for the 'old' flow to include an action trigger to shrink from fullscreen to original height?
it appears that some publishers now develop their websites with AMP only, including for desktop. In this case amp-consent doesn't fit well (especially because of the 60vh fixed height). What do you think about it ? Is this a common case ? If so, we should consider a responsive design for amp-consent or allow CMPs to access the window width to adapt their design.
Good point! Question: for desktop, amp-consent doesn't fit well because 60vh is too small?
If so, my impression is that the new lightbox design could help solve this issue. Definitely a scenario worth considering when we are implementing.
Once the lightbox flow has been implemented, would you see a need for the 'old' flow to include an action trigger to shrink from fullscreen to original height?
I think it depends the schedule because we will probably use the new lightbox flow for our TCF v2 version if available on time. In any case, we obviously won't apply the new lightbox flow to our TCF v1 version so my request is mainly for this one. But I think if the old flow remains an option, that feature would be useful for UX.
Good point! Question: for desktop, amp-consent doesn't fit well because 60vh is too small?
60vh is too big for desktop. You can see one of our publishers having integrated our CMP here. Definitely doesn't look great on desktop :) I can adjust CSS within the iframe but since I can't access the window width, I'm not able to set the right amp-consent's initialHeight (something like 30vh should be fine).
Thanks
But I think if the old flow remains an option, that feature would be useful for UX.
I see, that sounds good. It sounds like a reasonable request, I will see how we can plan around this work.
I'm not able to set the right amp-consent's initialHeight (something like 30vh should be fine)
You should be able to set the initialHeight to 30vh. Are you having trouble doing so?
When you send the ready signal to the AMP parent doc from your consent iframe, you can add the initialHeight
field to 30vh
(see here for an example).
You should be able to set the initialHeight to 30vh. Are you having trouble doing so? When you send the ready signal to the AMP parent doc from your consent iframe, you can add the initialHeight field to 30vh (see here for an example).
Yes I can, that's what I do when I set the 60vh initialHeight. But I would like to set the good initialHeight depending on the window width, for example 60vh for mobile and 30vh for desktop.
I would like to set the good initialHeight depending on the window width
Ahhh I understand. Thanks for the clarification. Not sure if we have something like this/how feasible this would be to add to amp-consent. I will look into it.
Also, @tla-sirdata, for the new Lightbox flow/TCFv2 support, do you imagine that you will upgrade all of your publisher's pages to it? Or would it be up to the publisher or some other factor? Essentially I am trying to determine who chooses what consent framework is used for each page.
If we manage to design a nice and readable UI with the old flow despite all information we have to disclose, I think we'll provide publishers with the two flows and it will be up to them to choose. And on the first place we will decide the default flow (probably the lightbox flow). About the transition between TCFv1 and TCFv2, it will also be up to publisher to upgrade as it requires their partners to be ready too.
@zhouyx is there any information on what is expected in the consentMetadata as part of reject and accept actions?
@vyoungnyc We are still working on the consentMetadata
field that can be sent via external iframe consent flow (accept/reject) and via checkConsentHref
endpoint.
So far, we will be accepting 3 fields within consentMetadata
:
Please let us know if you have any questions of concerns about these.
Also with the updates to webkit ITP and increasing difficulty for 3p cookies being set in an iframe, would it be possible to accept a field that persists for a cmp specific uuid for that particular user/consent string?
It would be much easier and dependable to be able to pull up the consentuuid and retrieve the up to date consent settings for that particular user without depending on cookies or working around itp.
@vyoungnyc Would something like CLIENT_ID
, work in this case? You could get this value for the user and then have it sent to your backend to help calculate and update consentString.
CLIENT_ID could work if it can be included in the clientConfig passed to the checkConsentHref endpoint as well as the promptUISrc iframe. that way we can match it up on our backend and provide the up to date consentString
CLIENT_ID
is created under the amp-consent
scope. Which means it's unique for the same user, not unique among different CMPs. In theory there's only one <amp-consent>
allowed on the page, so there will only be one CMP. But in the case where a publisher switch to another CMP, the id is going to be reused. Does this work for you?
While we could expand the support to clientConfig
. CLIENT_ID
is only supported in the promprUISrc
and the checkConsentHref
today. Example usage here
https://github.com/ampproject/amphtml/blob/081a69f1f7a6416946f780bed31f01db70588d15/extensions/amp-consent/0.1/cmps.js#L31-L35
While we could expand the support to clientConfig. CLIENT_ID is only supported in the promprUISrc and the checkConsentHref today. Example usage here
Additionally, within checkConsentHref
we send the current stored consentString and consentState which, along with the CLIENT_ID could help you determine new consentString and consentState. Does this work?
The Transparency and Consent Framework (TCF) is an effort to manage user consents, and have all parties understand the users consent information from regulations like GDPR.
Consent Management Platforms (CMP) helps sites to collect and manage user consent information in the TC data format. Today AMP allows a CMP to integrate with some limitations. The
<amp-consent>
CMP integration was designed to work with any framework that is based on a consent string format.CMP scripts are executed in an CORS iframe that’s controlled by AMP instead of included as first party script. Because AMP handles the lifecycle of the iframe, CMPs don’t have flexibility on when to show its UI, or has the ability to provide mandatory APIs for vendors to get their TC data.
What has Changed
In the past year, the IAB TCF v2.0 has been widely adopted by most vendors. Unlike a year ago when we first designed the feature, now we don’t see the necessity to support other TC framework. This allows AMP to work together with CMPs to understand the TC data and TCF v2 APIs.
With more regulations like GDPR, we are seeing the increasing complexity for site owners to manage user consents. Thus we are seeing the trend to have more CMPs come into the area. That brings a lot more feature requests and many ask for feature parity between AMP and their non-AMP integrations.
Goal
The goal of the design revise is to come up with a solution so that
Proposed solutions
Option 1
Allow CMP script to run in the background, it will still run in a CORS iframe, but it will be initiated at pageload time and always run in the background.
This is very similar to the CMP, non-AMP pages integration as much as possible.
Details
__tcfapi()
method and send postMessage to CMP iframe__tcfapiLocator
directly using postMessages.Pros:
Cons
Option 2
This is the current approach. Pass the entire TC data consent string using AMP provided APIs
Design Details
context.initialConsentValue
getConsentPolicyInfo()
methodCONSENT_STRING
macroPros
Cons
To decide if we need to change the design, we'd like to get feedback from CMPs, Ad vendors, Analytics vendors and Publishers. We are leaning towards Option 2 (The existing option) for performance reason. If so
cc @ampproject/wg-analytics @ampproject/wg-ads