FLOC performance assessment

[lbdvt]

On Jan 25 Google published a blog post stating:

By creating simulations based on the principles defined in Chrome’s FLoC proposal, Google’s ads teams have tested this privacy-first alternative to third-party cookies. Results indicate that when it comes to generating interest-based audiences, FLoC can provide an effective replacement signal for third-party cookies. Our tests of FLoC to reach in-market and affinity Google Audiences show that advertisers can expect to see at least 95% of the conversions per dollar spent when compared to cookie-based advertising. The specific result depends on the strength of the clustering algorithm that FLoC uses and the type of audience being reached.

These results will be used by many in the industry -advertisers, publishers, ad tech providers- to define their strategy regarding the retirement of third-party cookies in Chrome. A FLOC 95% replacement of conversions per dollar spent may render all other alternatives useless, including TURTLEDOVE.

In order to fully understand the scope of these results, could you please publish the related study that led to this conclusion?

[deepakr-google]

We have some response here: https://github.com/google/ads-privacy/blob/master/proposals/FLoC/Floc-live-experiments.md

[Pl-Mrcy]

As explained in @ablanchard1138's email, we have been already aware of this document for a while. We find it widely insufficient to reach:

1. a good understanding of the experiment and its conclusion
2. a point where the experiment would reproducible by other actors

To my knowledge, you didn't publish anything new since. ​ To fully grasp what to make of the widely publicized 95% figure, we must understand in details what the exact experience was and get more granular results. Could you publish in a "peer-reviewable" document, the full specifications of the test and the results? ​ These specifications should cover these areas and answer these questions, among others:

• Scope of the test
• Test span and test population ratio. On what volume of traffic was the test conducted?
• Cohort details:
  • Clear user-cohort assignment process (when is the FLoC assigned?; browsing history used; refresh frequency; etc.)
  • Minimum cohort size
• While 3P cookies were removed from the audience building aspect of the pipeline, what other role did 3P cookies keep playing in the test arm (bidding models, delivery limitations, etc.)?
• Detailed results
  • Number of conversions & conversions per dollar
  • Results projected over several dimensions (histograms, etc.)
  • Ex post cohorts analysis (size, etc.)
  • etc.

This would be extremely useful and will help us drive the conversation forward.

[deepakr-google]

Thank you for your feedback. We are working on the details. May take a few weeks stay tuned.