WICG / turtledove

TURTLEDOVE
https://wicg.github.io/turtledove/
Other
542 stars 239 forks source link

Questions about efficiency and fairness of interestGroupBuyers Implementation #981

Open hanishi opened 10 months ago

hanishi commented 10 months ago

I am writing to address a critical aspect of interest group-based advertising concerning the interestGroupBuyers field in the ad auction process. This issue centers around the matching efficiency between a user's stored interest groups and the DSPs included in the interestGroupBuyers list by the seller (typically a publisher or SSP).

Core Concerns

Matching Efficiency:

There seems to be a potential inefficiency when there is a mismatch between the user's browser-stored interest groups and the DSPs listed in the interestGroupBuyers. This situation is akin to a "high cache miss" scenario where the system fails to find a relevant match, leading to inefficiency in ad auctions.

Single Entity Manipulation:

The current framework allows both the seller and buyer sides to manipulate which DSPs are included in the interestGroupBuyers list. While this can optimize ad relevance, it raises questions about fairness and the potential for market distortion.

Discussion Points

How are Others Addressing This Issue?

Are there existing strategies or solutions that have been effectively addressing the efficiency concern in matching interest groups with interestGroupBuyers? What best practices can be adopted to enhance the matching process while maintaining user privacy?

Appropriateness of Single Entity Control:

Is it appropriate for a single entity (seller or buyer) to have significant control over the composition of interestGroupBuyers? What are the potential risks of this control in terms of market fairness and competition?

Alternative Models or Suggestions:

Can alternative models or mechanisms balance efficiency, fairness, and privacy more effectively? Can a more dynamic, real-time system for populating interestGroupBuyers be considered?

Impact on Smaller Players:

How does this system impact smaller publishers or DSPs in terms of participation and competitiveness in the ad auction ecosystem?

I am seeking the group's insights, experiences, and suggestions on this matter. Collaborative discussion can lead to a better understanding of these challenges and the exploration of potential solutions. Any shared experiences, data, or innovative approaches to similar issues would be highly valuable.

michaelkleber commented 10 months ago

It's certainly true that a seller running a Protected Audience auction gets to decide what buyers may submit bids. But that seller needs to exchange money with that buyer; they must have some sort of business relationship for the whole system to work. How could a seller not be responsible for listing the buyers they work with?

hanishi commented 10 months ago

Thank you for your insights on the importance of established business relationships in the selection of interestGroupBuyers. I'd like to delve further into the operational aspect of this system, specifically concerning the efficiency of matching user interest groups with the appropriate DSPs.

While the necessity of business relationships is clear, I'm particularly interested in how we can optimize the matching process to reduce instances where user interest groups don't align with the listed interestGroupBuyers. This misalignment seems to potentially lead to a high rate of "cache misses," thereby affecting the efficiency of the ad auction system.

Are there existing strategies or proposals for dynamically adjusting the list of interestGroupBuyers based on real-time data or user behavior trends? How might such a system work while respecting privacy norms and ensuring data security? What approaches can be taken to balance the efficiency of the matching process with the need to maintain established business relationships? Could there be a hybrid model that accommodates both aspects effectively? How might these solutions be adapted or implemented in our current context? I believe that addressing these points could significantly enhance the operational efficiency of ad auctions.

I look forward to further insights and suggestions on this matter.

michaelkleber commented 10 months ago

I think I still don't understand the essence of your concern. Why would an SSP's interestGroupBuyers not simply list all the DSPs they work with? In that case there is no privacy risk and no balancing needed, right?

hanishi commented 10 months ago

Thank you for your input.

I was reading the following two documents:

Protected Audience (formerly known as FLEDGE) Integration and Testing Guide for SSPs

Indicate Protected Audience ad renderability

From what I understand, the SSP sends a BidRequest with an indication that there might be a relevant interest group from a DSP present in the user's browser, though the SSP itself does not have direct knowledge of this. Here's a breakdown of this process and its implications that I understood.

SSP Sends BidRequest with a Signal:

When an SSP detects an impression opportunity (e.g., a user visits a webpage with an ad unit), it sends out a BidRequest to potential DSPs. This BidRequest includes a signal indicating that the impression is from a browser that may contain interest groups relevant to the DSP.

DSP's Decision to Participate:

Upon receiving the BidRequest, the DSP must decide whether to participate in the potential browser-held auction. This decision is based on the likelihood that the user’s browser contains the DSP's interest groups.

The DSP does not know directly whether its interest groups are present in the user's browser at the time of the auction. This approach does align with privacy-centric advertising models, where user-specific data is not explicitly shared with external entities like DSPs or SSPs. The browser acts as a privacy gatekeeper, holding and managing interest group information, which is the whole purpose of the framework.

While this method upholds privacy, it introduces challenges in auction efficiency. DSPs must make bidding decisions with incomplete information, which could impact the relevance and effectiveness of served ads. When sending BidRequests, SSPs include signals suggesting the possibility of a match with a DSP's interest group. However, they do this without knowing the specific interest groups in the user’s browser. This approach requires SSPs to balance the number and variety of DSPs they include in the auction process, managing efficiency and potential ad relevance.

Your suggestion to include all DSPs an SSP works with in the interestGroupBuyers list is doable. However, this might not fully address the essence of the concern: auction efficiency.

hanishi commented 10 months ago

I forgot to mention about the cost of network traffic usage is substantial, not to mention server instances to keep up with.

hanishi commented 10 months ago

Your perspective has prompted me to reflect on my part, leading me to a self-conclusion I would like to share with you for confirmation.

My understanding and conclusion

SSP's Discretion in Buyer Selection:

SSPs can choose which DSPs to include in the interestGroupBuyers. This choice is based on existing business relationships and the historical performance of the DSPs.

SSP's JavaScript tag and real-time decision-making:

The SSP's JavaScript tag embedded on the publisher’s site is crucial in dynamically deciding which DSPs to involve in each ad auction. This decision-making is based on the context of the ad request and potentially the relevance of the DSPs’ ad inventory to the content.

Balancing traffic among DSPs:

Despite the presence or absence of specific interest groups in a user's browser or whether the auction is PAA eligible, SSPs have always been tasked with balancing ad requests among DSPs. This balancing act has ensured fairness and efficiency in the ad auction process; if they have not, the issue might be attributed to the SSP's optimization strategy or capabilities and not specific to the usage of the interestGroupBuyers in a way that I have discussed and what I had concerned about.

My understanding reconciles the operational aspects of SSPs. However, I would appreciate your thoughts or any other thoughts to confirm if my conclusions align with the industry's current practices and trends.

I really appreciate your expertise in this area, and I look forward to any further clarification or perspective you can provide.

Thank you.

michaelkleber commented 10 months ago

I understand what you are saying, but...

Despite the presence or absence of specific interest groups in a user's browser or whether the auction is PAA eligible, SSPs have always been tasked with balancing ad requests among DSPs.

If an SSP chooses to list a DSP among its interestGroupBuyers and that DSP has zero interest groups on the user's device, then the fact that the SSP listed them does not matter at all — essentially no resources are consumed on the browser or in the auction.

It seems to me that this puts on-device auctions in a very different place from RTB auctions, with inevitable server-to-server connections involved in an SSP offering a spot to a DSP.

hanishi commented 10 months ago

Thank you once again for sharing your expertise and perspectives. Here are my conclusions.

Impact on DSP strategies

Shift from contextual to interest-based targeting:

DSPs are likely to redirect their focus and budget from contextual targeting, which relies on the web page's content, to interest-based targeting, which utilizes user interest group data. This shift is in response to the loss of detailed user tracking capabilities provided by third-party cookies.

Adapting to new data sources:

DSPs must adapt to using different data sources, like first-party data or aggregated and anonymized data, to inform their interest-based targeting strategies.

Challenges for SSPs

Reduced control in ad selection:

SSPs face a reduction in direct control over the ad selection process. In-browser auctions, driven by the user’s browser using locally stored interest group data, diminish the SSP’s ability to influence which ads are shown.

Managing a more complex ecosystem:

SSPs will need to manage relationships with DSPs and publishers in a more complex ecosystem, where the mechanics of ad delivery are less transparent and more decentralized.

Revenue predictability concerns

Uncertain revenue projections:

The predictability of ad revenue becomes more challenging in an environment where ad selection relies heavily on browser algorithms and the variable nature of interest groups.

I would appreciate hearing your thoughts regarding this transition.

hanishi commented 10 months ago

If an SSP chooses to list a DSP among its interestGroupBuyers and that DSP has zero interest groups on the user's device, then the fact that the SSP listed them does not matter at all — essentially no resources are consumed on the browser or in the auction.

Regardless of the on-device auction process, SSPs will still need to send BidRequests for contextual auctions. These requests are essential for DSPs to respond with their bids, even when they are not aware of whether their interest groups are present in the user’s browser. This process incurs network traffic and associated costs for the SSP, even if the on-device auction itself does not. As DSPs increasingly focus on interest group-based targeting, SSPs’ reliance on historical data for predicting which DSPs to send BidRequest might become less effective and achieve the same level of efficiency in the auction process as in the past, when detailed tracking data was available, will become a significant challenge in this new paradigm at least until they find new ways to gather and analyze data that comply with privacy regulations and the constraints of on-device auctions. This might involve focusing more on aggregated data, contextual signals, and permissible first-party data. This inefficiency could lead to a scenario where SSPs must broaden their BidRequest dispatches to include all DSPs they work with, potentially increasing operational costs initially.

To maintain efficiency, SSPs could initially include all associated DSPs in interestGroupBuyers. However, they would only actively notify and involve DSPs in perBuyerSignals if those DSPs start showing a pattern of winning bids in browser-held auctions. This approach relies on analytics and winning bid patterns to optimize the process, reducing the guesswork and potentially minimizing unnecessary network traffic. That said, if querying perBuyerSignals from all DSPs to be included in the interestGroupBuyers is mandatory, the cost associated with these queries becomes an inevitable part of the auction process.