Open johnwilander opened 4 years ago
Ping @michaelkleber @csharrison.
I don't have any problem with specifying a Javascript API, but I would be concerned if we dropped the HTTP technique entirely to avoid the need for more third-party Javascript running in privileged contexts. I wonder if it's possible for the two techniques to stay in sync by establishing a 1:1 correspondence between e.g. a JS init dictionary and a structured header dictionary.
To ensure we have the same security properties, the JS API should probably only be callable in the reporters origin too.
We don't want more third-party scripts in the first-party context either which is why we intend to have an optional restriction on the calls to this JavaScript API where cross-site scripts will not be able to use the API. There are no browser engines with JavaScript source tracking yet, but there may be in the future and we should acknowledge that to disincentivize sites from importing cross-site scripts to do conversion calls.
Regardless, we have no immediate plans of deprecating the legacy method with tracking pixels. However, the spec should indicate that it is a legacy thing that's bad for privacy, performance, and power. It also runs the risk of being blocked by content blockers, in which case there will be no report.
I agree we should restrict the JavaScript API to the first-party context.
I have a feeling it will help us design the legacy trigger format by exploring a modern JavaScript API.
I don't quite understand your desired end-state. Is it:
In our design we wanted to support third party reporting endpoints to avoid advertisers / publishers needing to do their own analytics. For those cases it seems good to allow permission delegation (possibly via something like Feature Policy) so those endpoints have control over whether a conversion occurred or not.
I don't quite understand your desired end-state. Is it:
- Conversions can only be registered by the first party (i.e. the advertiser)
- The advertiser cannot delegate conversion registration to any party (i.e. their ad-tech)
We're talking about triggering or signaling a conversion here, right? Such as "the customer put a product in the shopping cart."
We want there to be a JavaScript API for this and that browsers are allowed to restrict calls to that API to only true first-party scripts. Such a restriction will protect the integrity of conversion signaling and make it hard for malicious or fraudulent third-party scripts (in first- or third-party contexts) to trigger false conversions. False conversions could be done to 1) falsely consume ad clicks for competing websites where ads are displayed or 2) to falsely trigger ad click attribution reports to prop up the numbers for your own ad clicks.
In our design we wanted to support third party reporting endpoints to avoid advertisers / publishers needing to do their own analytics. For those cases it seems good to allow permission delegation (possibly via something like Feature Policy) so those endpoints have control over whether a conversion occurred or not.
The JavaScript API would not in itself restrict where reports could go. As a straw man, consider this:
navigator.adAttribution.convert(conversionValue, ["search1.example", "search2.example", "news1.example", "news4.example", "gamingSite.example"]);
The call above says "A conversion happened with value conversionValue
and I want it to trigger attribution reports for any pending ad clicks on search1.example, search2.example, news1.example, news4.example, or gamingSite.example because that's where I'm running ads for this product."
You could also do this:
navigator.adAttribution.convert(conversionValue, ["*"]);
… which would mean "A conversion happened with value conversionValue
and I want it to trigger attribution reports for any pending ad clicks on any website that took the user to my site."
Ah OK I see. I still have two small concerns:
fetch("https://ad-tech.com/log-conversion?<metadata>").then((r) => {
navigator.adAttribution.convert(<something based on the response>);
});
@michaelkleber may have other thoughts here though.
I generally agree with @csharrison that we want to make the new API as easy as possible to adopt (as with basically any API!), and that advertisers will have it much easier if they can just tell their ad tech partners to do the job for them.
@johnwilander I haven't heard concerns before about malicious 3rd-party scripts having 1st-party context on advertiser web sites and faking conversion reports. Is that a real worry that advertisers have expressed?
I'm all in favor of the 1p getting to control what capabilities 3p's get delegated. But an API that distinguishes "true first-party" seems like a lot of work, so I'd like a better sense of the size of the problem.
Ah OK I see. I still have two small concerns:
- Making advertisers themselves log conversions incurs a large adoption cost (whereas allowing ad-tech to do this on advertiser's behalf requires orders of magnitude fewer total code changes).
We are aware of this but our position is that longterm, we need to move away from reliance on third-party resources for this kind of customer data and reporting. And when I say we, I mean browsers. If the advertiser makes a deal with a service provider to process their data on the backend, that's on them, and something they have to explain to their customers.
One might argue that merely placing a third-party script on a website effectively is such a deal and should be counted as a delegation of power. But we've seen again and again how third-parties repurpose their scripts to do new things without even an opt-in signal from site owners. For instance, there was a case with an analytics script that suddenly started processing cross-site ad attribution and leveraged first-party storage for it, presumably to circumvent tracking protections. Another case is a social networking script that suddenly started scanning the DOM for PII such as email addresses and sending them to their own server, apparently with no specific opt-in from site owners.
- Ad tech operating on behalf of the advertiser may be in a better position to choose the right metadata values (i.e. they already log unattributed data, and can translate that data into the most efficient metadata possible). This is partially mitigated by advertisers doing something like:
fetch("https://ad-tech.com/log-conversion?<metadata>").then((r) => { navigator.adAttribution.convert(<something based on the response>); });
@michaelkleber may have other thoughts here though.
I generally agree with @csharrison that we want to make the new API as easy as possible to adopt (as with basically any API!), and that advertisers will have it much easier if they can just tell their ad tech partners to do the job for them.
@johnwilander I haven't heard concerns before about malicious 3rd-party scripts having 1st-party context on advertiser web sites and faking conversion reports. Is that a real worry that advertisers have expressed?
This is based on the flood of information on fraudulent behavior in virtually all aspects of online advertising. There's no reason not to believe that every avenue for fraud will be explored.
I'm all in favor of the 1p getting to control what capabilities 3p's get delegated. But an API that distinguishes "true first-party" seems like a lot of work, so I'd like a better sense of the size of the problem.
What I'm arguing is that the spec is written so that browsers are allowed to restrict calls to the conversion API to true first parties.
Fair enough — you are quite welcome to be even more pessimistic about fraud than I am.
If we're relying on browser limitations that aren't possible today, then I would prefer to leave the door open to an API that can only be called by "real" first-party code or by explicit delegation of that power, e.g. with some markup like the iframe-inspired <script allow="adAttribution" ...>
.
Sure. I'll draft something. I know Feature Policy is a thing in Chrome land. Could that be a fit?
Currently we don't have a lot of per-origin feature restrictions for script running in the same execution context. Feature policy is really all about per-document permissions which is usually how API permissions are scoped.
@clelland, have you considered feature restriction like this before (only same-origin script can use), and extensions to feature/document policy that selectively allow 3p scripts to use the features?
I hadn't been considering that specifically, if only because of the difficulty of the problem (if a 3p script triggers a 1p script to document.write a new script with 3p-supplied content, how do we attribute actions from that new script? What if the 3p adds or overwrites methods on objects that were originally created by the 1p?)
If we could lock down a mechanism for that kind of attribution, then feature policy might be a good fit for that; it's a similar delegation problem to allowing powerful features in selected 3p frames.
Enforcing true 1p script is difficult and I'm not sure it increases the fraud resistance (though it does increase incovenience for merchants).
We're already including the zero-adoption tracking pixel workaround, so it won't be an adoption barrier, but adding a cleaner API might not work out if we don't make it usable in the way most adtech integration works.
@othermaciej can you point us to this zero-adoption tracking pixel workaround? The industry works off an advertiser being able to audit their publisher supply with independent vendors.
Why is this change being proposed? What problems, specifically, is it solving? Is it audience tracking? Well that won't be able to happen without cross-site tracking, so this doesn't do anything new in that front.
This is akin to an advertiser not being able to independently audit a store with how many visitors it and how many bought its product. The reason that advertisers don't just accept publisher numbers is because advertisers don't really trust those numbers and want independent auditors.
Is the problem being able to attribute a conversion to an IP? Can't the law be used to enforce that advertisers, like publishers, don't use IP for any purpose?
@johnwilander I haven't heard concerns before about malicious 3rd-party scripts having 1st-party context on advertiser web sites and faking conversion reports. Is that a real worry that advertisers have expressed?
This is based on the flood of information on fraudulent behavior in virtually all aspects of online advertising. There's no reason not to believe that every avenue for fraud will be explored.
Why would an advertiser (w third party conversion tracking) have the incentive to inflate conversions? In fact the opposite, if they don't want to pay a publisher. Publishers do a lot of click-stuffing on platforms like mobile in-app. The point is that first party publishers have the most incentive to inflate attributions. That's why 3rd party audits exist. Maybe I misunderstood how this mechanism does better at preventing conversion fraud.
The only place where a 3rd party conversion tracker inflates numbers is in mobile-browser->in-app conversions because it's probabilistic. And regular conversion tracking is fully deterministic and advertisers audits vendors by checking that those conversions work by triggering the conversion flow manually.
There's also so many different types of conversion rules built into those systems, web browser devs would now be burdened with supporting those. I think something akin to https://github.com/csharrison/aggregate-reporting-api but with an open system. Yet in an open system that can guarantee data usage and privacy through self-enforced cryptographic verification.
@therevoltingx To get some background on the discussion, please first have a look at the blog article and the spec linked at the bottom of https://github.com/WICG/ad-click-attribution
You will see that there is a "legacy" tracking pixel version and a "modern" JS API version of the conversion trigger.
There is a similar proposal from the Chrome team, part of the privacy sandbox, with (IIRC, and last time I checked) support for third-party ad tech vendors, more bits for the click id (more entropy...), support for view thru conversions, etc.
Now, as explained here, they are trying to align the two proposals, and this thread is just one of the many issues being discussed.
@abebis ok so after having reviewed the specification and blog I think they're woefully misguided and don't show a whole lot of knowledge about advertising. I think calling it "zero adoption" is borderline lying.
Let's focus, for example, on click based attribution.
"Traditionally, such attribution has been facilitated by user identifying cookies sent in third-party HTTP requests to the click source." - This is just completely wrong. It's frightening, because only view-through conversions work that way but view-through conversions are not even mentioned.
Here's how conversions work:
This literally has zero to do with cookie or even users. What are you all so worried about leaking here?
A conversion is a true multi-party transaction. The publisher shows the advertiser's creative, the user clicks on the advertiser's creative, the user converts on the advertiser's property.
Furthermore, this mechanism is not verifiable or auditable by the advertiser. And even furthermore, putting conversions in the hands of publishers makes fraud a much bigger problem.
We don't have to use entropy or be probabilistic, that's a bad assumption and not novel. If you take a look at the crypto space for instance, these proposals seem quite outdated.
And even more, because this is not against the law pubs and advertiser will simply move to server-to-server conversion tracking. So this won't actually do anything but make the lives of pubs and advertisers harder.
The click includes contextual data and the advertiser generates a 'click id' which is a randomly generated id which is then passed to the url of the landing page.
Don't you mean the publisher (or ad network), instead of the advertiser?
In my understanding, there are two types of ID parameters sent to the landing page (LP) through link decoration.
Click ids are unique for each click and thus for each user. Because of the risk that click ids might be used to build cross-site user profiles, ITP (since ITP2.1) and other browsers have started restricting "click id"-based tracking by limiting the lifetime of client-side storage to 1-7 days, or (like Brave) by completely removing known click id parameters from urls.
Ad ids on the other hand are not unique for each user (as long as an ad is not displayed/targeted to a single user...) and thus cannot be (easily...) used to build cross-site user profiles.
Ad ids seem to be "acceptable" for browsers, at least according to Mozilla anti-tracking policy
Acceptable Example:
A site wishes to track conversions after a user interacts with an ad. The site can annotate the landing page URI of outbound advertisements clicks with information about which advertisement was clicked and from which publisher. When a user later completes a conversion action, third-party code from the site transfers information about the advertisement that led to the conversion back to its servers, such that an aggregate number of conversions can be computed. This is acceptable under our policy because it does not involve the creation of a user profile.
Unacceptable Example:
Similar to the example above, site A wishes to track conversions. But unlike that example, site A decorates all outbound links with a unique identifier that is mapped back to an individual user. A user clicks on a search result for site B and later purchases a product from site B. When this purchase is completed, the user’s unique identifier is sent back to site A’s servers to record the purchase. Due to the fact that site A is building a profile of user purchases, this approach goes beyond conversion measurement and would be a violation of this policy.
But it is hard for browsers to make the distinction between click and ad ids, so ad id-based tracking has also been impacted by ITP.
This literally has zero to do with cookie or even users.
url parameters are typically stored in a cookie so that it can be retrieved later on the CV page. And click ids can be mapped to unique users on the publisher/ad network.
What are you all so worried about leaking here?
They/we are worried about click ids being used to build cross-site user profiles. You can argue that the publisher/ad network already knows the LP url of the click anyway, but I guess conversions are more valuable that visits. (Also, the Google version of the PCM API uses a high-entropy click ID that can be mapped to a single user, which adds to the (my) confusion 💫 )
That being said, I must admit I don't see the use case for the PCM API in a world where click/cv measurement can still be done through ("acceptable") link decoration. click info stored in document.cookie and PCM are both restricted to 7 days (1 day for document.cookie in some cases, but not enough to justify the switch to PCM).
I guess within 2 years browsers will further restrict document.cookie (treated like a session cookie?), making PCM comparatively more attractive. Then,
@abebis
Thanks for your detailed response. Please see my answers inline.
Don't you mean the publisher (or ad network), instead of the advertiser?
No, but i certainly see the confusion here. I was thinking specifically of "Publisher Ad Server" based systems and not search or Facebook ads.
A "Publisher Ad Server" is a product like Google's Ad Manager (formerly DoubleClick for Publishers), the most popular publisher ad server in the world.
Publishers use a publisher ad server by adding a javascript snippet to every place in their website where they want to show an ad.
With "Publisher Ad Server" based advertising, there's two sub-categories:
Insertion Orders are old-school but still widely used in the industry. It's great for premium publishers but does not scale.
With Insertion Orders, an advertiser will send a creative (e.g. png, video, etc.) and a "landing page url".
The publisher ad server will also support "macros" with which the landing page/click url will be "decorated" with information about the context from the publisher.
https://support.google.com/admanager/answer/2376981?hl=en
So an example landing page url, given to the publisher by the advertiser, would look like:
https://pepsi.com?buyercampaignid=0x123&buyergroup=0x768&adid=%%ADUNIT%%&lineitemid=%eaid!
The publisher ad server expands the macros to their final value before being served and rendered into a page
Within programmatic, you have different types of creative types video, image, HTML ad, etc.
Programmatic is great because it allows reaching more and smaller publishers at scale.
The contents of the ad (e.g. the html content) is delivered to the publisher ad server via bid response. The publisher ad server then injects the html content into the publisher ad slot, usually an iframe.
With static banners and videos, the bid response will contain an explicit click url. With rich media, the click url is usually defined within javascript and can vary wildly depending on the creative. (e.g. multiple landing pages.)
In these cases the buyer will give their bidder a landing page url, the landing page url is then "decorated" with information via DSP-side macros.
https://support.google.com/displayvideo/answer/2789508?hl=en
So an example click url, given jointly by the bidder and the buyer via a merged url is:
https://pepsi.com?buyercampaignid=0x123&buyergroup=0x789&itemid=${CAMPAIGN_ID}
The bidder expands the macros to their final value before being included in the bid response.
At the end of the day, regardless if it's a publisher ad network, an ad network, or programmatic.
The landing page url is defined by the buyer.
A good example is going to weather.com and clicking on any of the ads there.
Typically a buyer will use what's known as a "Buy-Side Ad Server" to track the performance of a campaign.
This is what most buyers think about when it comes to "tracking."
So pepsi.com will really look like: buysideadserver.com, which will then redirect to pepsi.com.
A buyer can 'decorate' buy-side adserver click urls with buy-side ad server macros: https://support.google.com/dcm/table/6096962?hl=en
What does this mean? That all of these groups appear across different products that are used by different people.
For a marketer, they will be looking at the buy-side ad server. For the traffickers, they will be looking at campaigns from the dsp point of view. For a publisher, they will be looking at their publisher-side ad server.
Each of these has different concepts of groups, campaigns, creatives, placements, line items, insertion orders, etc.
Click IDs are generated by multiple parties in the click chain, EXCEPT for a publisher. Publishers typically do not know about clicks to a given creative. Just go to weather.com and click on any ad.
Consider the case of a single-page conversion landing page. Remembering that multiple-step conversions will pretty much never convert (at least not click-through.)
In those cases a cookie is neither needed, nor desired. This is the vast majority of cases.
In this case a conversion is counted by the buyer (really the buy-side ad server on behalf of the buyer).
This works by the buyer looking up the click id and figuring out which publisher, site, etc generated the conversion. THEN they tell the publisher of such a conversion.
Keep in mind that publishers very rarely get paid for conversions directly, rather they get paid for impressions. This is why they don't usually get clicks. A marketer upstream will be the one getting paid for delivering on conversions.
Click ids are unique for each click and thus for each user. Because of the risk that click ids might be used to build cross-site user profiles, ITP (since ITP2.1) and other browsers have started restricting "click id"-based tracking by limiting the lifetime of client-side storage to 1-7 days, or (like Brave) by completely removing known click id parameters from urls.
I certainly challenge this assumption, I do not think it's correct. First it's clear that advertisers do not care about an individual, but rather:
Secondly, please explain how a click id can be tied to a given indivdual. This is the core of my question. We should thus focus the conversation here, once we have the defined the base definitions.
url parameters are typically stored in a cookie so that it can be retrieved later on the CV page. And click ids can be mapped to unique users on the publisher/ad network.
This is not normal, this is an edge case and not what is typically done or should be done. Rather the landing page click is decorated with a click id. Then the click id/postback url is added to the form as a hidden field. Multi-step conversion flows have horrible conversion rates. Even then, it's usually done through recursive link decoration when navigating the steps.
"Similar to the example above, site A wishes to track conversions. But unlike that example, site A decorates all outbound links with a unique identifier that is mapped back to an individual user. A user clicks on a search result for site B and later purchases a product from site B. When this purchase is completed, the user’s unique identifier is sent back to site A’s servers to record the purchase. Due to the fact that site A is building a profile of user purchases, this approach goes beyond conversion measurement and would be a violation of this policy."
Sorry but all I have to say is WTF. "That's not how any of this works" :p But more seriously, this is not at how click id based conversions work. Why would site A ever know of a conversion on site B?
I still don't see how this improves on existing cross-site tracking restrictions. Also, again, this type of conversion tracking will simply move to server-to-server.
Personally, I think this is dogmatically overreaching and even imposing rules that go way beyond the strictest consumer privacy laws.
Brave simply has built-in ad blocking. But ad blocking is terribly overreaching and should certainly not be the norm. This is why it's so important to define what exactly "tracking" is to this group.
Thanks, not an ad server expert...
Secondly, please explain how a click id can be tied to a given indivdual. This is the core of my question. We should thus focus the conversation here, once we have the defined the base definitions.
Click ids are generated for each click, and thus can be potentially linked to a persistent id managed by the entity (ad server) that generated the click id: Google/Doubleclick for Google Search/Display Ads, Facebook for... Facebook Ads. Persistent id could be the logged in user id (for Google/FB) or cookie (for Doubleclick?)
The risk is well summarized in this thread
UTM are used for a website's own anayltics while fbclid will link users actions on a website to advertising done on facebook and report on it within fb. UTMs just track where a user came from, fbclid will track who a user is and match it back to fb ads.
Click ids can be tied to a user, unlike the campaign/ad group/creative ids you are mentioning.
I certainly challenge this assumption, I do not think it's correct. First it's clear that advertisers do not care about an individual, but rather:
Yes. But Google and Facebook and other ad networks care (to "personalize" ads). The advertiser can read the click id but since it will change for each click it won't leak any information about the user.
@othermaciej can you point us to this zero-adoption tracking pixel workaround? The industry works off an advertiser being able to audit their publisher supply with independent vendors.
He's referring to "Legacy Triggering of Ad Click Attribution" in this proposal.
Why is this change being proposed? What problems, specifically, is it solving?
Do you mean the modern triggering API discussed in this issue or the whole PCM proposal?
Is it audience tracking? Well that won't be able to happen without cross-site tracking, so this doesn't do anything new in that front.
This is akin to an advertiser not being able to independently audit a store with how many visitors it and how many bought its product. The reason that advertisers don't just accept publisher numbers is because advertisers don't really trust those numbers and want independent auditors.
Is the problem being able to attribute a conversion to an IP? Can't the law be used to enforce that advertisers, like publishers, don't use IP for any purpose?
We are not considering IP addresses here. Did you read the proposal? It's here: https://wicg.github.io/ad-click-attribution/index.html I think a lot of what you're asking about is answered in there.
@johnwilander I haven't heard concerns before about malicious 3rd-party scripts having 1st-party context on advertiser web sites and faking conversion reports. Is that a real worry that advertisers have expressed? This is based on the flood of information on fraudulent behavior in virtually all aspects of online advertising. There's no reason not to believe that every avenue for fraud will be explored.
Why would an advertiser (w third party conversion tracking) have the incentive to inflate conversions?
That's not what I was saying. The merchant should be in control of when conversions are triggered since when an attribution request goes out, the corresponding ad click is consumed and no further reporting will be possible.
With full API access by all scripts in the first-party context, click-source-2.example can maliciously trigger early conversions for click-source-1.example to flush its pending ad clicks and inject bad data while doing the correct conversion reporting for itself.
In fact the opposite, if they don't want to pay a publisher. Publishers do a lot of click-stuffing on platforms like mobile in-app. The point is that first party publishers have the most incentive to inflate attributions. That's why 3rd party audits exist. Maybe I misunderstood how this mechanism does better at preventing conversion fraud.
The only place where a 3rd party conversion tracker inflates numbers is in mobile-browser->in-app conversions because it's probabilistic. And regular conversion tracking is fully deterministic and advertisers audits vendors by checking that those conversions work by triggering the conversion flow manually.
There's also so many different types of conversion rules built into those systems, web browser devs would now be burdened with supporting those. I think something akin to https://github.com/csharrison/aggregate-reporting-api but with an open system. Yet in an open system that can guarantee data usage and privacy through self-enforced cryptographic verification.
@abebis ok so after having reviewed the specification and blog I think they're woefully misguided and don't show a whole lot of knowledge about advertising. I think calling it "zero adoption" is borderline lying.
Let's focus, for example, on click based attribution.
"Traditionally, such attribution has been facilitated by user identifying cookies sent in third-party HTTP requests to the click source." - This is just completely wrong. It's frightening, because only view-through conversions work that way but view-through conversions are not even mentioned.
Here's how conversions work:
- A user click on an advertiser's creative (e.g. Pepsi). The click includes contextual data and the advertiser generates a 'click id' which is a randomly generated id which is then passed to the url of the landing page. (e.g. pepsi.com?click_id=0x123)
- The user lands on the advertiser's site, when they complete an action the 'click id' is passed through via the conversion tracker to the advertiser
- The advertiser looks up the click via the click id to get the context (e.g. which ad, from which pub, etc.)
Click IDs in so called link decoration were traditionally used in conjunction with third-party cookies and since many third-party cookies are now blocked in browsers with tracking prevention, that way of reporting conversions doesn't work anymore. The practice of leveraging first-party scripts to instead store click IDs in first-party cookies was done in an effort to circumvent third-party cookie blocking AFAIK.
This literally has zero to do with cookie or even users. What are you all so worried about leaking here?
A conversion is a true multi-party transaction. The publisher shows the advertiser's creative, the user clicks on the advertiser's creative, the user converts on the advertiser's property.
Furthermore, this mechanism is not verifiable or auditable by the advertiser. And even furthermore, putting conversions in the hands of publishers makes fraud a much bigger problem.
We don't have to use entropy or be probabilistic, that's a bad assumption and not novel. If you take a look at the crypto space for instance, these proposals seem quite outdated.
And even more, because this is not against the law pubs and advertiser will simply move to server-to-server conversion tracking. So this won't actually do anything but make the lives of pubs and advertisers harder.
@therevoltingx, if you want to discuss PCM and ad click attribution in general, please file individual issues. This issue is tracking a JavaScript API that's functionally equivalent to the tracking pixel redirect, nothing else.
thank you ...... I got the point
I believe the JS API would be a great addition including for, but not limited to, developer convenience. However, Apple PCM efforts should not worry about anything fraud-related since that is out of scope (if not, then PCM scope should be limited). The fraud is both
Apple should not worry about fake data sent by the ad-tech to the advertisers/publishers, and shouldn’t try to govern (or execute) its opinion in this space - in order to prevent encroachment of internet freedom in addition to preserving privacy.
PCM mentions Modern Triggering of Ad Click Attribution which is a placeholder for a JavaScript API with which ad click destinations can programmatically signal conversions and desired ad click matches as opposed to redirected pixel requests. This functionality is desired because it doesn’t require a network load, it doesn’t require a redirect, it doesn't require third-party requests which leak IP address, referrer et cetera, and it doesn't multiple loads and detections to trigger attribution reports to multiple ad click sources.
How would you like to proceed on specifying this JavaScript API?