Debug tool like Professor Prebid for Protected Audience Auctions

[pm-harshad-mane]

Multiple advertisement slots exist on the page where ad auctions occur among various participants. This auction setup is quite intricate, involving multiple players bringing diverse signals. At times, debugging the setup becomes necessary to gain a precise understanding of the auction's proceedings. The Prebid ecosystem, having evolved significantly, offers numerous methods and tools for bugging auction setups. Prebid auctions provide a range of event handlers to disseminate information about these events. List of the available events is maintained here, https://docs.prebid.org/dev-docs/publisher-api-reference/getEvents.html

Protected audience auctions prioritize privacy at their core. Consequently, individual participants are unable to access data points of other participants. Unlike analytics partners within Prebid, no player can join the auction solely to collect data that would help consolidate auction-related data points.

More details about the Prebid Chrome extension, Professor Prebid, can be found here: https://docs.prebid.org/tools/professor-prebid.html Following are some screenshots of the open-source debug tool developed by the Prebid community:

Configuration of each Ad unit / Ad Slot:

Details of the bid received:

Timelines of the bid response:

The tool also creates an overlay on the actual ad-slot on the page:

Considering the privacy-centric nature of the protected audience auction, no external script on the page can gather all the internal details of the protected audience auction so it would be helpful for the community if Google Chrome team develops some tool to display all such details in the Chrome Dev Tools. We will be happy to contribute for the development efforts.

Due to the privacy-focused design of the protected audience auction, external scripts present on the page are unable to access comprehensive internal details of these auctions. It would greatly benefit the community if the Google Chrome team could create a tool within Chrome Dev Tools to showcase these otherwise inaccessible details. We're eager to contribute to the development efforts required for this initiative. This tool would serve as a valuable resource, offering visibility into the intricate workings of protected audience auctions without compromising their privacy-centric structure.

Following is a list of features we would like the tool to have:

• Display the ad-slot level details.
• List of participants: component sellers, buyers, interest groups with respective configs
• Details of the real-time signals: trusted bidding signals, trusted scoring signals
• Auction configs and other signals
• Received bids per seller with renderingURL, ad-metadata, k-anonymity score.
• List the network calls separately apart from the usual Network Tab
• Timeline of events

[JensenPaul]

A few followups on this issue:

1. We discussed this for a while on the WICG Protected Audience call last week, notes here
2. There was an ask about Chrome extensions talking to DevTools. The answer is yes, this is possible, see here
3. We're hoping to add a better runAdAuction() integration with the DevTools -> Application -> Storage tab soon.

[dmdabbs]

Hello @pm-harshad-mane. See Kevin Lee's PR for example DevTools protocol code to read PAAPI info: https://github.com/privacysandbox/privacy-sandbox-demos/pull/269

Related Chromium issue https://issues.chromium.org/issues/41489627

[dmdabbs]

DevTools code moved here: https://github.com/GoogleChromeLabs/trust-safety-demo/pull/56

[pm-harshad-mane]

Thanks @dmdabbs!

[pm-harshad-mane]

Debug Information mentioned in https://github.com/WICG/turtledove/issues/430 can be useful to be presented in this debug UI.

[morlovich]

So this landed in https://github.com/GoogleChromeLabs/trust-safety-demo/tree/main/protected-audience/samples/devtools-extension/src now, with Chrome side available fully since 123.0.6274.0

The example should be usable as-is (with a very "programmer UI" definition of usable), but it's basically meant to be sample code that anyone can use to help them get started if they want to build this sort of thing. (It's Apache 2.0 licensed).

[pm-harshad-mane]

Thanks @morlovich

[pm-harshad-mane]

@morlovich, how can we view the inputs and outputs for each generateBid / scoreAd function call? Additionally, we're interested in viewing the responses from the KV servers, which serve as input to the aforementioned functions.

[morlovich]

Hmm, good questions. You can /almost/ get the inputs by using the special breakpoints, but the timing is wrong; it should be easy to add new ones right before running of the function rather than top-level, though might also need to do the "debug(generateBid)" thing at that point to get it exactly right. I will try this out and get back to you. Also beware cumulative timeouts, and it is likely to slow things down.

Not sure if there is an easy way of seeing the return value, though the bid events have the important stuff; I could probably also add some score events?

I am not sure if any of the Network. and Fetch. mechanisms for intercepting things will work, I suspect not. We could probably hook them up. though I will need to check whether it makes it easier for people to do nefarious things. Just fetching the URL again will almost always give you the right stuff... except if it changes :(

[morlovich]

For first one, https://chromium-review.googlesource.com/c/chromium/src/+/5387607/1/content/services/auction_worklet/bidder_worklet.cc has the needed Chrome change, and sample modification to the extension stuck in a comment.

This produces something like this:

### Response:{"result":{"type":"object","value":{"0":{"ads":[{"metadata":{"adName":"default-ad"},"renderURL":"https://protected-audience-demo-dsp.web.app/ads/default-ad.html","renderUrl":"https://protected-audience-demo-dsp.web.app/ads/default-ad.html"}],"biddingLogicURL":"https://protected-audience-demo-dsp.web.app/bid.js","biddingLogicUrl":"https://protected-audience-demo-dsp.web.app/bid.js","enableBiddingSignalsPrioritization":false,"executionMode":"compatibility","name":"tv","owner":"https://protected-audience-demo-dsp.web.app","trustedBiddingSignalsSlotSizeMode":"none","useBiddingSignalsPrioritization":false},"1":{"isControversial":true},"2":{"windowInnerHeight":1014},"3":null,"4":{"adComponentsLimit":40,"bidCount":12,"forDebuggingOnlyInCooldownOrLockout":false,
(truncated)

on one of the demo sites.

There is probably a way of getting the return value too with some strategic stepping, since Debugger.paused has a 'returnValue' field in CDP inside the callframe structure: https://chromedevtools.github.io/devtools-protocol/tot/Debugger/#type-CallFrame ... though that seems a little messy to fit into house the thing is structured.

[jlquaccia]

Hi @morlovich, thanks for the response! I tested out the code changes from https://chromium-review.googlesource.com/c/chromium/src/+/5387607/1/content/services/auction_worklet/bidder_worklet.cc locally on my end, however I am unable to access the code inside of this if statement during PA auctions (as I can't seem to get the if statement criteria to be met):

if ( method === 'Debugger.paused' && source.targetId && params.data?.eventName === 'instrumentation:beforeBidderWorkletGenerateBid' ) { let targetId = source.targetId; let childDebuggee = {targetId: targetId}; let frameId = params.callFrames[0].callFrameId; chrome.debugger.sendCommand( childDebuggee, 'Debugger.evaluateOnCallFrame', { callFrameId: frameId, expression: 'arguments', returnByValue: true, }, (response) => { console.log('### Response:' + JSON.stringify(response)); chrome.debugger.sendCommand(childDebuggee, 'Debugger.resume', {}); } ); }

This functionality should be available in the latest version of Chrome right? I have Version 123.0.6312.86 (Official Build) (arm64)

Seems the "Debugger.paused" event isn't getting triggered after setting the beforeBidderWorkletGenerateBid event breakpoint?

[morlovich]

It requires that CL, so it's not in any released version, sorry.

[jlquaccia]

@morlovich ok, thanks for the heads up :+1:

[jlquaccia]

Hey @morlovich just a quick follow up question, what does "CL" refer to?

[morlovich]

Changelist. Roughly the same as what github would call "pull requests", just without an underlying pull operation involved (I think it's probably originally Perforce terminology, but I am not sure).

[pm-harshad-mane]

@morlovich does that mean the change is yet to be released? If so, what is the target release schedule?

[morlovich]

It's not landed; I first have to make sure that my colleagues more familiar with ad industry feel it's not likely to be used to do nefarious things.

[pm-harshad-mane]

Thanks for the update @morlovich !!

[pm-harshad-mane]

@morlovich until you get feedback from your colleagues, just to limit the access to the API... can you make this API available only for code executing through Chrome Extension ?

[jlquaccia]

Hi @morlovich, one question we have been curious about lately was around detecting when PA auctions end. I know there are win and topLevelBid events that can be listened for during an auction, but since these events don't always occur, is there a way to listen for when a PA auction has ended?

Also, just wanted to follow up on Harshad's question here as well about the code to view the inputs and outputs for each generateBid / scoreAd function call

@morlovich until you get feedback from your colleagues, just to limit the access to the API... can you make this API available only for code executing through Chrome Extension ?

[pm-harshad-mane]

Happy to share PubMatic's debug tool to help debug Protected Audience auctions... Blog: https://pubmatic.com/blog/unlocking-easy-debugging-for-googles-protected-audience-auctions/ Chrome Extension: https://chromewebstore.google.com/detail/pubmatic-pa-debugger/mhjjhcjpihlohkimmlceppocmijooegm