jonasz
commented
4 years ago Hi Andrew,
Thank you for the suggestions and the data. I am happy to see, PLTD would allow you to retain a high fraction of clicks and impressions at reasonable product-level audience size thresholds.
Still, I understand it is an important use case for NextRoll to be able to serve rare products in ads.
To comment on your points 2., and 3., I think it is important to understand the product-level proposal as: "policy mechanisms should be applied to products, not to collections of products", and not as a discussion in favor of minimal audience thresholds, or in favor of delays in calls to fetch-ads.
Product level Turtledove opens up the possibility to drop audience thresholds altogether, as described at https://github.com/jonasz/product_level_turtledove#auditability, and we'd be happy to explore this idea further. I was wondering, would that satisfy your use case?
- and shuffles them before they are passed to the "template" web bundle. I don't think this is feasible for performance reasons. I understand the desire to avoid ordering-based pranks. However, there are a couple of reasons I object to this. One is that Facebook shared results that removing user information from their ranking algorithm dropped publisher revenues by 40-50%.
Note that the product-level proposal only mentions shuffling the items after they've been chosen for the particular user (so it affects the ordering, not the selection of items). Are you sure this is what FB did in the study you're referencing? In our experience the drop in revenue would be orders of magnitude lower. Of course the drop is there and, if possible, at RTB House we'd prefer not to shuffle, but it's not a deal breaker for us.
To answer the point 1. - Of course, the product-level approach should be understood as an extension to TD; "single-product", "no-product" and "static" ads would still be supported in PLTD. Perhaps the confusion comes from the definition of "ecommerce" and "product"? We meant it as defined at https://en.wikipedia.org/wiki/E-commerce.
appascoe
michaelkleber
benjaminsavage
As discussed last week, here's my list of proposed changes to product-level TURTLEDOVE, along with some graphs and discussion.
Proposed Changes
We argue that in ecommerce it is the product, not the creative (a collection of products), that would be the most natural conceptual unit behind Turtledove mechanisms.I would strike this. Products are a natural fit for retail advertisements, but may not be suitable for B2B services, general branding, etc. NextRoll itself has many clients that continue to use static creative. I would consider this proposal as supplemental to TURTLEDOVE, with an admittedly important use case.I would like to add that requests to
../.well-known/fetch-adsaren't necessary when the browser is on the advertiser's site. We can eliminate minimum audience thresholds by writing web bundles at this time. Moreover, this speed to send web bundles to the browser is critical for such product advertising use cases as the "identity recommendation" algorithm which recommends recently viewed products. (It's also useful because ads delivered to users who have more recently been to an advertiser's site are significantly more valuable, so eliminating the delay preserves publisher revenue.)../.well-known/fetch-adsis still useful as it provides an opportunity to update recommendations or the creatives intended to show in the browser hours or days after the browser has been to the advertiser's site.Below, I've attached graphs that demonstrate how minimum audience thresholds can drastically affect the performance of other types of clients beyond larger retail players. This is another reason they should be a fallback plan than the primary mechanism for ad/product selection.
...but conceptually, we just need to get rid of very rare products and then just run the old algorithm.Very rare products are important for ad delivery in small campaigns that are run by merchant platforms. NextRoll is a partner with both Yelp and Rakuten, and the graphs below demonstrate the effects of including and excluding them in our analyses. We should try to solve for these advertisers' use cases.and shuffles them before they are passed to the "template" web bundle.I don't think this is feasible for performance reasons. I understand the desire to avoid ordering-based pranks. However, there are a couple of reasons I object to this. One is that Facebook shared results that removing user information from their ranking algorithm dropped publisher revenues by 40-50%. Second, is that at NextRoll, we blend recommendation algorithms to hedge bets, though some products are clearly superior to others; in the situation of a carousel ad, the first item seen can significantly affect engagement with the ad (from scrolling through to other products, to overall CTR).I'd expect ordering-based pranks, such as assembling text, should be handled by a sufficient audit through an SSP. Audits are part of the ecosystem today to increase trust, and I see no reason this should change.
Graphs and Discussion
At NextRoll we duplicated the graphs in this proposal with our own data. As I said, we're partners with marketplace platforms such as Yelp and Rakuten (our two largest partners). Their merchants purchase advertising through the marketplace, and we're one of the channels these marketplaces leverage. As such, NextRoll frequently runs campaigns with modest audience sizes and very small product catalogs. The behavior and performance of these marketplaces and campaigns are so radically different than an advertiser who comes to us directly, that we frequently need to separate them out for analysis. The graphs below should demonstrate this (Rakuten identified as "R" and Yelp identified as "Y"):
As can be seen, our retained CTR drops off a cliff for many advertisers with an audience threshold of 30 when marketplaces are included. Note that our fraction of retained events actually improves when they're including. Essentially what's happening is there are lots and lots of merchants with such modest deliveries, that they don't account for a significant number of events in our dataset, but this large number of advertisers would be adversely affected by an audience threshold of 30.
This is just one of the reasons why we here at NextRoll are arguing so much for the ability to add web bundles while on the advertiser's site without any audience size thresholds. These small clients should be able to participate in the overall digital ad ecosystem so they can attempt to grow their brands. This comes at no cost to privacy due to first-party cookies sticking around.