Dev Tools Request: New Application Section Panel for Auctions with "Fledge Internal" Logging, Log Levels, etc

[thegreatfatzby]

Introspecting Chromium Space Sandbox Decisions

We've talked a lot about debugging various Privacy Sandbox elements, but one on which I'm not sure what the current state of thinking is, is the ability to debug/inspect client side auctions w/r/t the actions the auction framework is taking internally (i.e. not user space code) at various steps. This includes things like verifying delegation permissions, inclusion of buyers overall and their specific IGs, running prioritization dot products and deciding what to include/exclude, and choice or running/not-running updateUrl.

Right now in "user space" code we can do normal console.log, debugging of user space scripts, and even for now debugging of the bidding/scoring/reporting functions. However, Fledge being a highly configurable auction framework, many choices/decisions/paths are taken by fledge internal code based on configurations, which are not directly observable in those ways. As a developer this type of observability is critical, as often a problem is due to your own config or API usage choices being subtly wrong, and so the framework does exactly what you told it to but just not what you wanted it to.

Some recent examples where I made a mistake and had to figure it out:

• I wasn't clearing IGs in a test, and so when I edited code to modify properties and re-ran it, the old IGs were updated and some values in priortySignalsOverrides I didn't want hung around and messed with my dot-product'ing. I was tired, tunnel visioned, etc; eventually I queried the IG sqlite db, and realized what was going on.
• An IG wasn't being included in an auction even though it was being loaded. I was building some code and going step by step just wanted to see the bidding function work, and so had some logs in it. In my haste I left my ads element was empty, and therefore the framework chooses not to let it bid.
• Delegation permission denials despite seeing my server serve the file...turns out missing header, which I had set previously in another place, knew about, but skipped over in copying some code over.
• updateUrl was called immediately after join, and so when I called the FOT updateAdInterestGroup I didn't see it update.

In all of these cases it was both a) completely my fault and knowable from docs but also b) hard to get any information as to why since I can't spelunk around the Chromium code with a debugger, log lines, etc*.

As more and more developers start using these APIs for critical work, this type of observability will be needed to scale successfully.

Proposal

So my proposal is to provide the type of "internal visibility" that lots of MVC, ORM, etc frameworks offer. I'll start with simple logging of selected internal events to a new panel in the Dev Tools --> Application --> Application section, let's call it Privacy Sandbox, that would work something like this:

• Offer the standard 5 log levels, error, warn, info, debug, trace.
• Have warn selected by default.
• Chromium code starts to include some "dev tool facing logging", with the idea being the chromium devs have to choose to have something included in the dev tools. The function would be pretty standard log lines with content, a log level, and maybe a category. (Something JSON based with properties that are filterable would be neat too).
• This should include events from anything sandbox related, but for now I'll start with asking for pieces relevant to the issues called out above: Delegation file fetching, importing, rejection reasons. Priority calculations and filtering. IG inclusion decisions. updateUrl not called b/c called too recently.

* I recognize I technically could do this, but either a) there's a tool I don't know about to make this "easy" or b) it would involve a custom build and I hope we're agreed that's not a good solution for when lots of devs start to use this.

[patmmccann]

Related, we'd love something like the chrome://topics-internals tab that has this info: https://github.com/WICG/turtledove/issues/974#issuecomment-1885124912

[thegreatfatzby]

Want to add Private Aggregation API cases to this, being able to see the "application logs" from the decisions the Chrome PAg and when it's planning to send things would be really helpful in trying to debug...and also, seeing if it's making a decision based on a chrome://flag, as I'm not clear if I'm setting conflicting settings, etc.

Thinking about this a bit more, it actually might be a better idea if it was possible to have a "Chromium Log" "section" in the console log alongside the current error/info/etc logs. Philosophically it would treat this as a single auctioning application, and practically being able to line up those logs (as if debugging a single app) would be really useful, i.e. "I see X happen in my bidding fuction, followed by Y in Chromium, followed by Z issue occurs in scoreAd".

[dmdabbs]

Hey @thegreatfatzby Isaac, I wonder whether the DevTools protocol examples morlovich posted recently might provide a start on this event tracing: https://github.com/WICG/turtledove/issues/937#issuecomment-1997926783 and a basis for enhancement.