Prebid.less Proposal

[bwschmidt]

Type of issue

Intent to Implement

Description

Prebid.less Proposal

MARKET SITUATION & PROBLEM STATEMENT

Over the last several years, Prebid.js has cemented itself as the defacto header bidding solution. However, maintaining and configuring Prebid has proven cumbersome for many publishers. There is a consistent need to update Prebid core code along with a multitude of adapters. Secondly, there is a huge setup and maintenance cost related to keeping bidder parameters and adunit mapping up to date with every demand source a publisher wants to use. The market’s response to this has been a myriad of managed solutions. These solutions offload much of this complexity including keeping the source up to date and managing adunit configuration.

There are other demand aggregation solutions in the space without these problems, most notably Google Open Bidding and A9 TAM. These solutions integrate completely transparently to the publisher, requiring no adapters and no adunit mapping. This offloads the vast majority of the setup work to demand sources and saves publishers critical time and resources. This is accomplished by:

• Standardizing on OpenRTB
• Requiring exchanges to handle adunit mapping

These solutions have proven that demand sources (e.g. Exchanges, SSPs) can and will implement standardized implementations in order to not lose out on monetization.

OPPORTUNITY

By standardizing on OpenRTB and requiring demand sources to handle ad unit mapping for publishers we feel that Prebid can close a critical benefit gap between Google Open Bidding and A9 TAM ultimately driving the following benefits:

• Increased publisher adoption of Prebid from publishers lacking development and adops resources
• Improved publisher revenue from increased experimentation
• Reduction of publisher time & resources expended on maintaining Prebid
• Reduced effort required for publishers to comply with future industry standards and regulations (e.g. CCPA, supplychain object)

CONCEPTUAL DESIGN

1. Prebid.less participants would be required to implement OpenRTB endpoints for both client and server-side requests ⋅⋅ The client-side endpoint would be required to read IP and cookies from the header ⋅⋅ The endpoint would utilize standard OpenRTB, and any applicable global extension fields used by Prebid.js
2. Prebid.js and Prebid server would no longer have bid adapters, but instead have 1 single adapter that calls each demand source in the exact same way.
3. Publishers should be able to select which demand sources are called for which adunit on the page. ⋅⋅ Publishers should be able to opt demand sources in or out of sensitive fields in the OpenRTB request, e.g. user.ext.eids
4. Demand sources will be required to utilize dfp_adunit_code, e.g. “/123/atf” and the div_id, e.g. “div-gpt-ad-123-0” in order to map the request to their specific inventory

RELEASE MILESTONES

• Build a standardized adapter for both Prebid.js and prebid-server.
• Have a migration period where publishers can opt demand sources into these new adapters while keeping some on their current adapter setups. ⋅⋅ Continue to support bidder params as an extension field during this period
• Once there is mass adoption, begin removing demand source adapters.
• Lastly, remove the need for client side bidder parameters.

[bretg]

Thanks for thoughtfully launching this conversation @bwschmidt. There have been internal discussions in the general direction... it's a good general direction. We'd like for you (and other interested parties) to join Prebid.org to help us flesh out and drive a vision.

The conversations so far have revolved around several key points:

1) Lighten the browser footprint with javascript that just knows how to gather key page parameters, contact Prebid Server, provide targeting to the ad system, and render results. It would be a totally separate codebase. 2) Document Prebid Server better so more companies can spin it up 3) Allow adslot name to use the existing "Stored Request" feature to find parameters. (Note this requires an interface for managing mappings of adslot to bidders and params) 4) Support a "generic" openrtb server-side adapter to cover those entities that don't already have a Prebid Server adapter

One stumbling block though, has been "Why don't more people utilize Prebid Server already?" We think it's most likely either cookie match rate or the fear of reducing match rate.

Other potential issues:

• Prebid Server doesn't come with an interface for stored-request mapping. The various server hosts have rolled their own.
• Server-side analytics is currently not nearly as rich as client-side analytics
• This approach wouldn't be nearly as customizable, tweaking bidrequests, bidresponses, inspecting the bid cache, etc.
• Prebid Server isn't easy to set up and run. Many pubs would want to utilize a managed service. But people like Prebid.js because it's inherently transparent. Utilizing a vendor's Prebid Server is not quite as transparent. We have ideas to address this, but will take some time.

Responding to specific items in your suggestion:

1. Requiring client-side protocols to be pure OpenRTB isn't going to work, IMO. Can't imagine trying to get 250+ busy and already stressed companies to spend months changing this.
2. We think the focus should be server-side so the beefy features like currency conversion, GDPR, size-mapping, etc can get out of the browser.
3. "Generic" openRTB is a bit of a unicorn even server-side. There are 40+ server-side adapters even now, and almost all of them have snowflaky little deviations/additions. Maybe 1 or 2 are straight up. But this is ok, and not a major part of your proposal. Server-side adapters exist for most major bidders and they don't change that often.

But in any case, this has been very much a back-burner/long-term project at this point -- we're focused on TCF, Programmatic Guaranteed, long form video, etc. To kindle interest in throwing resources at a major project like this, it would help to have some energetic prebid.org members helping drive it.

[richccalkins]

@bretg thanks for the thoughtful and thorough response. @bwschmidt and I both work at OpenX and were on the PMC call today. We'd be happy to drive this idea and open to feedback on the best way to go about doing this. We would have brought it up today, but wanted folks to get a chance to digest the proposal first. Is the best next step to get it on the agenda for 2 weeks from today? Blast out the link in the prebid-js channel to get more feedback in the comments here? Both? Neither?

[bretg]

Even though this idea is really bigger than Prebid.js, I think that's currently the best forum to start the discussion since it's javascript. I imagine it will make sense to carve the discussion out into a separate meeting at some point. It would help drive the conversation if someone were to write up some specific requirements, taking into account important details like GDPR, User ID modules, ad server integration, etc.

[GeneGenie]

Even though openrtb has recommended itself as a good protocol, ad almost standard s2s type of integration. it would take couple more years for most companies to adopt this tech. E.g look at vpaid and how long it took before it came to production, and how long AdTech companies couldn't provide IAB compliant publisher api after that.
Of course prebid.js could become "ClientSide BidSwitch 2.0" but this will take much more efforts from prebid team to validate endpoints, and make sure everyone is compliant. I can't imagine this type of prebid with more then 20 adapters =) I believe that OpenX and Xandr follow standards, but there are not so much companies that implement IAB protocols with such precision.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[khatibda]

if not switching entirely to a universal openRTB adapter (quite like this broad direction), is there a roadmap to iteratively reduce the footprint of each/all of these adapters? some are getting to be over 5kb, and there has got to be a way to tighten the scope of what's needed or allowed in an adapter, on a feature by feature basis.

as for prebid server, it simply doesn't perform as well as client side on a per adapter basis, for most people. yet. cookie match rate (major) and extra network hop (minor). identity solutions can hopefully replace cookies over time, but chrome isn't there yet. but can the server-side solution patch this with client side pixel sync and cookie pass-throughs per bidder (adapter lite or different kind of module?)

[bretg]

iteratively reduce the footprint of each/all of these adapters

Sort of. The new 'module rules' draft puts controls around loading external code and phoning home for analytics. But adapter size isn't a rule. The last time I heard anyone mention about overall wrapper size is that the difference between a code size of 100KB and 300KB isn't really that big for most clients.

However, as far as I know, no one's shared a detailed breakdown of where browser time is spent. That would be welcome.

can the server-side solution patch this with client side pixel sync

There is already a client-side /cookie-sync happening. PBS-Java has implemented the "Cooperative Cookie Sync" described in https://github.com/prebid/prebid-server/issues/924, but PBS-Go has not.

cookie pass-throughs per bidder

I think this already covered. PBS cookie-sync works like this:

If PBS is configured in PBJS, /cookie-sync is always fired. PBS uses a "match table" cookie called "uids" that's in the domain of the PBS host company. The results of the cookie-sync redirects is that every supported bidder's ID is stored in the uids cookie. e.g.

{
    "uids": {},
    "tempUIDs": {
        "adnxs": {
            "uid": "4722255122219375043",
            "expires": "2020-06-01T20:01:23.569Z"
        },
        "triplelift": {
            "uid": "9328941297032053459",
            "expires": "2020-06-01T20:01:26.923Z"
        },
        "ix": {
            "uid": "XlV6w9HM6LYAAHx2YJ4AAACZ&476",
            "expires": "2020-06-01T20:01:25.422Z"
        },
        "yieldmo": {
            "uid": "ge515bd6c7da71cdc98a",
            "expires": "2020-06-01T20:01:25.926Z"
        },
        "adform": {
            "uid": "1707054018971720697",
            "expires": "2020-06-01T20:01:24.569Z"
        },
        "brightroll": {
            "uid": "y-S8Fq5QZ1lwWKPeXdoZ9vSeZx47maINFrJeY53pDtokA2FlaPmwvrJg--",
            "expires": "2020-06-01T20:01:24.001Z"
        },
        "consumable": {
            "uid": "ue1-sb1-aa634f4b-d618-4378-b8c3-9baa56dcb91a",
            "expires": "2020-06-01T20:01:22.874Z"
        },
        "pubmatic": {
            "uid": "2ECE1904-7EB2-4C38-98A4-38E97535AA9C",
            "expires": "2020-06-01T20:01:51.234Z"
        },
        "rubicon": {
            "uid": "KACWYIER-P-59CH",
            "expires": "2020-06-01T22:07:27.192Z"
        },
        "pulsepoint": {
            "uid": "dcxvyKqDV5VV",
            "expires": "2020-06-01T20:01:19.738Z"
        },
        "sovrn": {
            "uid": "bad97f98b08c9204fe6b9826",
            "expires": "2020-06-01T20:01:18.936Z"
        },
        "openx": {
            "uid": "f1f4ac13-99f8-46da-82f8-b52c29b378e0",
            "expires": "2020-06-01T20:01:19.156Z"
        }
    },
    "bday": "2020-05-18T20:01:18.934Z"
}

We're happy to field any suggestions you have towards improving the PBS match rate.

[khatibda]

The last time I heard anyone mention about overall wrapper size is that the difference between a code size of 100KB and 300KB isn't really that big for most clients.

omg this is terrifying if true.

thank u for the detailed cookie explanation! helpful to understand.

[acework]

Wanted to chime in here - from our perspective, a jump from 100kb to 300kb is a no-go - we get push back from our sites for increases much less than that. We're interested in ways for Prebid to get slimmer.

[mike-chowla]

One of things that would help to cut down adapter size is the ability to compile out code for formats the publisher is not using. PubMatic's adapter code size would drop about 40% if we could exclude native code at build time. Most publishers don't use native so being able to exclude it in those cases would be big win.

Another idea is to post list of list of adapter code size with largest first which would create a wall of shame of sorts. I say this knowing full well PubMatic's adapter could use some trimming down and this won't look great for us.

[mike-chowla]

One idea to improve Prebid Server match rates: The Prebid Server match rate gets better the more sites are running a vendor's Prebid server. What if all member companies run their Prebid servers under a prebid.org domain and thus sync their server cookies into that domain. There would be a bunch of details to work out. We'd probably need to use delegated DNS sub-domains so everyone can managed their server setup as they need to.

[patmmccann]

this is being implemented as #8562