Google ads testing Iba with privacy preserving signals raw data

[iab-france]

Hi Google Ad team,

Thank you for providing us with detailed analysis and results of your testing on the Topics API.

To further understand the impact of the removal of 3PCs, could you also disclosed the raw data from your testing? More specifically, could you provide us with actual spend, impressions, clicks and conversions numbers, for each dimension from table on page 6?

For clarity, we would like to get metrics above for GAD, with and without AI-powered optimization, and for DV360, with and without AI-powered optimization, for both test and controls groups.

Thank you!

Alliance Digitale France

[ardianp-google]

Hi, thanks for your enquiry.

Unfortunately, we are unable to share the raw data as it contains business sensitive information. If you could share the goals behind your ask, we are happy to partner and provide alternatives that can help accomplish the goal.

[iab-france]

Thank you for your prompt response.

On behalf of Alliance Digitale, a trade organization dedicated to assisting the industry in navigating emerging trends like Privacy Sandbox, we greatly appreciate your collaboration. As we approach the general availability phase, it becomes increasingly crucial for us to equip our members with comprehensive information regarding testing results. Therefore, we kindly requested access to raw data to ensure the accuracy of our analysis. Below, you will find the analysis we have conducted thus far, which we are eager to compare and validate against the actual data.

We have reviewed the white papers you provided and would like to address some observations we made regarding the results tables for both Google Display Ads (GAD) and DV360. Specifically, we noticed that while advertiser spend and conversion per dollar metrics exhibited minimal impact (-1.9% and -0.4% for GAD, +0.19% and -1.9% for DV360), the Click Through Rate (CTR) and Conversion Rate were significantly affected (-8% and -1.8% for GAD, -1.16% and -5.64% for DV360).

Initially, we encountered difficulty in reconciling these metrics, as CTR and Conversion Rate are expected to directly influence Conversion per Dollar metrics. However, upon further consideration, it became apparent that the reason advertiser spend and conversion per dollar were only slightly impacted in comparison to CTR and Conversion Rate was due to the disparity in impressions delivered between the test group and the control group. In other words, while the CTR and Conversion Rate may have decreased, the actual number of clicks and conversions could have remained unchanged due to the higher volume of impressions in the test group.

Using the provided Spend, CTR, Conversion Rate, and Conversion per Dollar impact metrics, we were able to deduce that a decrease of 5% to 10% in CPM (Cost per Thousand Impressions) could account for the differences observed between Spend and Conversion per Dollar metrics versus CTR and Conversion Rates. If this deduction holds true, it implies that the implementation of Topics instead of third-party cookies, limited solely to the scope of the experiment (i.e., only iba), had a 5% to 10% impact on CPM. Such an impact could potentially translate into a 5% to 10% reduction in revenue for publishers, which raises concerns considering the relatively narrow scope of the experiment.

In order to validate this assumption, we kindly request your assistance in providing the industry with the raw data. We understand that releasing such information may not be feasible; however, if possible, we would greatly appreciate access to CPM metrics for control groups versus test groups, or alternatively, impressions-level data for test groups versus control groups. Obtaining this additional information would greatly aid our analysis and enable us to reach a more accurate understanding of the observed results.

Thank you in advance for your attention to this matter. We look forward to your response and the opportunity to further explore these findings.

Best regards, Alliance Digitale France

[anderagakura]

@ardianp-google Even if according to the doc "We recommend the results to be interpreted as directional indicators rather than precise estimates of 3PCD impact on IBA on Google’s display network. As a reminder, note that all other products (e.g. measurement, remarketing) retained 3PCs and we applied treatment only to Google buyside plaorms in this experiment", providing more info would help especially with the timeline going faster and faster.

[rushilw-google]

Hello, thank you for your detailed response. We look forward to our continued collaboration on the Topics experiment and further Google Ads testing of the Privacy Sandbox. We recognize the intent behind the analysis to evaluate the impact of Topics API to publisher revenue. As we share the metrics below, we ask that special attention be paid to the considerations also accompanying the metrics.

In the Topics experiment, the CPM (Cost per Thousand Impressions) for Interest Based Advertising (IBA) impressions decreased by 4.38%. Google’s Buyside also had a 3.74% increase in the total number of impressions served in Treatment relative to Control. As a result, the Buyside revenue for IBA was not significantly impacted. We believe our Sellside counterparts can best shed light on how these stats would impact publisher revenue holistically and we ask them to weigh in here.

We do want to contextualize that when evaluating the revenue impact of 3PCD to publishers, it is important to remember that Interest Based Advertising is just one of many channels of demand for their inventory. We are happy to further discuss our response and support the goals of Alliance Digitale France, thanks again for your questions.

[steveswan-google]

Thank you all for the discussion, I'd like to share thoughts from the Google Ad Manager and AdSense teams.

Ad Manager and AdSense appreciate the sentiment behind the ask from Alliance Digitale as we have a similar goal of ensuring publishers feel confident in their ability to sustain ad-supported businesses as we move together towards this milestone for our industry.

While we are supportive of the effort behind the IBA experiment and are encouraged by the direction of the results, we have shared the guidance with our own partners that this experiment cannot be used to conclusively forecast publisher impact from the deprecation of third-party cookies in Chrome for the following reasons:

• The experiment only measured the effectiveness of serving interest-based ads and therefore doesn't reflect the full scope of ad types from which a publisher can monetize their inventory, including contextual ads or remarketing.

• The experiment focused on open auction which is only one way a publisher can fill their inventory. Traditional reservation and programmatic direct, for example, are two sales channels that will become increasingly important for publishers with first-party data.

• The experiment continued to use third-party cookies for other use cases such as measurement and remarketing. When third-party cookies are no longer available in those scenarios, the market's ability to optimize campaigns towards publisher inventory will change in ways we're not able to reliably predict.

• The experiment was run in today's environment, where buyers are able to target using third-party cookies when available. Since many buyers will continue using third-party cookies until their deprecation, the removal of cookies in the experiment does not account for the real-world shifts in buyer behavior that will occur once third-party cookies are no longer available.

Starting in July 2023, Ad Manager will begin including Topics data on Chrome traffic wherever possible. While we’re making this change to better understand the API’s utility for our partners, it's important to remember that the Privacy Sandbox is just one of many tools that publishers will use to support their businesses in a privacy-first world.

First-party identifiers, for example, will play an important role in supporting key ad functionality for publishers of all sizes, while first-party audiences allow publishers to showcase the value they offer in helping marketers reach the right users.

In light of the upcoming testing modes announced by Chrome, Ad Manager is actively developing testing plans, in coordination with our partners, to account for these challenges. The goal of this testing is to share data and learnings so that our partners, and the industry as a whole, can have a more accurate picture of the potential impact to publisher revenue.

Best regards, Google Ad Manager and AdSense teams

[iab-france]

Thank you sincerely for your response! We would like to inquire about the computation details for the CPM KPI. It would be immensely helpful if you could provide us with information on both the spend and impressions aspects. If full details are not feasible, we kindly request the same level of granularity as provided for other KPIs in the white paper. Specifically, we are interested in differentiating the CPM KPI for GAD (Google Advertising Display) versus DV360 and analyzing results with and without AI involvement, similar to the breakdown available in the table on page 6 of the white paper.

In reference to your comment on the experiment's inability to definitively forecast publisher impact, we acknowledge that the test framework, as designed, presents limitations in drawing final conclusions regarding the impact on publishers. However, we believe it is essential to recognize that similar limitations apply to assessing impacts on the advertisers' side as well. Despite this, conclusions were drawn for the advertisers, albeit with proper warnings that the results may not fully represent the overall scenario but rather serve as indicative trends.

We appreciate your attention to these matters and look forward to further insights and clarifications on the CPM KPI computation and its associated breakdowns. Your support is crucial in our efforts to thoroughly evaluate and understand the impact of these experiments.

[rushilw-google]

Hello from Google and thank you for the questions on the Topics experiment. We are happy to provide the Alliance Digitale with additional information to better contextualize the experiment outcomes.

Please see the table below with CPM (Cost per thousand impressions) broken out by Google’s buying platforms and use of AI-powered optimization.

CPM Change	With AI-Powered Optimization	Without AI-Powered Optimization	Overall
Google Display Ads (GDA)	-5.18%	-5.26%	-5.03%
Display & Video 360 (DV360)	-3.53%	-2.61%	-3.33%

The CPMs were computed using the following formula:

Metric	Formula
CPM	[ Sum (spend) / Sum (impressions) ] * 1000