[Feature] Streamlined Buyer and Brand Id's

Slind14 commented 5 years ago

Feature

From a publisher perspective it would be great to get access to the buyer and brand id's. A couple of major exchanges (AppNexus, Rubicon, Index..) are already exposing this data but do not follow a common syntax.

Current Situation:

AppNexus (1.x pbjs bid object) Rubicon (1.x pbjs bid object) Index (bid respond, not transmitted to pbjs bid object)

Proposal

Major/Minor Version Optional streamlined/restful declaration for buyer, brand, landing page domain and additional dynamic information. Breaking Version (2.0) Network, Agency, Buyer, Brand and primaryCatId become mandatory.

Could look like:

meta: {
    networkId: 34,                          // Network/DSP Id
    networkName: 'Adform'                   // Network/DSP Name
    agencyId: 0,                            // Agency Id
    agencyName: 'direct',                   // Agency Name
    buyerId: 4994,                          // Buyer/Advertiser
    buyerName: 'Volvo',                     // Buyer/Advertiser Name
    brandId: 39342,                         // Brand/Product
    brandName: 'Volvo v40',                 // Brand/Product Name
    primaryCatId: 'IAB2',                   // Primary IAB Sub Category ID
    secondaryCatIds: ['IAB2-3'],            // Secondary IAB Sub Category IDs
    clickUrl: 'https://www.volvocars.com/nl/modellen/model/v40'   // landing page
}

Open to discussions

Are there any other variables/meta information which would/could be valuable for you?

mkendall07 commented 5 years ago

Probably should rename domain to clickUrl since that's closer to the industry name.

Slind14 commented 5 years ago

Probably should rename domain to clickUrl since that's closer to the industry name.

Does clickUrl always refer to the "landing page" or to the tracking pages in between as well?

mkendall07 commented 5 years ago

It depends. Some SSPs will use 2 separate URLS and some combine them (daisy chains style).

Slind14 commented 5 years ago

Updated split buyer into buyer and network. AppNexus reports both under buyer and most other exchanges report them separately. We want the lowest level of granularity and aggregate it on pub side if needed.

Slind14 commented 5 years ago

Updated added IAB Category ID

mkendall07 commented 5 years ago

@Slind14 I updated it a bit. Let me know what you think. I dropped creativeId and DealId because they already exist on bid and I don't think we need to make a breaking change just to move them.

Slind14 commented 5 years ago

Looking good. I have added network name for convince. For clarity and order I think it could be of benefit to include the deal and creative id in the meta data, be it by keeping a duplicate in the parent object or depreciating the duplicate with the next breaking version.

chefbenjamin commented 5 years ago

This is a great addition. coming from the publisher side, what are the chances this info can be added into the bids and winning bids scripts ? ie http://prebid.org/dev-docs/troubleshooting-tips.html#see-all-bids-in-the-console and http://prebid.org/dev-docs/troubleshooting-tips.html#see-all-winning-bids-in-the-console The ability to see this would help a great deal in previewing bad creatives and blocking crappy buyers within each of the SSP.

Slind14 commented 5 years ago

what are the chances this info can be added into the bids and winning bids scripts

A small addition to the script and you would be set. Though I highly recommend recording the data so you can track down and block bad ads more easily.

kelvin-chappell commented 5 years ago

It would be helpful to distinguish between advertiser and agency, where that information is available. So:

buyerId: 4994,                   // Advertiser/Agency
buyerName: 'name',               // Buyer Name

might become:

agencyId: 4993,
agencyName: 'agName',
advertiserId: 4994, 
advertiserName: 'adName',

Would that be feasible?

Slind14 commented 5 years ago

@kelvin-chappell updated it to reflect the additional dimension and added a simple example

mkendall07 commented 5 years ago

Small update to match what we did for the long form video competitive separation.

patmmccann commented 5 years ago

we'd really love it if hb_adomain was a possible targeting key

bretg commented 4 years ago

@Slind14 - is this issue ready to close?

@patmmccann - what is hb_adomain? Can you explain in a new issue?

Slind14 commented 4 years ago

Hi @bretg,

this is still pending proper implementation. Being added to the docs and becoming mandatory.

patmmccann commented 4 years ago

If each bidder set key hb_bidder_adomain with the advertiser domain, we would find that incredibly useful.

On Wed, Oct 16, 2019, 6:07 PM bretg notifications@github.com wrote:

@Slind14 https://github.com/Slind14 - is this issue ready to close?

@patmmccann https://github.com/patmmccann - what is hb_adomain? Can you explain in a new issue?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/prebid/Prebid.js/issues/3115?email_source=notifications&email_token=AAM25Z727BOFP27N2KN4JHTQO63DTA5CNFSM4FWRPVQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBOMZYA#issuecomment-542952672, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM25Z4DGFWARJ45CDBBUQ3QO63DTANCNFSM4FWRPVQQ .

bretg commented 4 years ago

Would you agree that hb_adomain_BIDDER could be optional as a targeting variable? We're already getting some pressure on how much data is going to the ad server.

@mkendall07 - making these fields mandatory or strongly encouraged seems like a candidate for 4.0?

patmmccann commented 4 years ago

Certainly makes sense for it to be optional

On Sun, Oct 27, 2019 at 11:13 PM bretg notifications@github.com wrote:

Would you agree that hb_adomain_BIDDER could be optional as a targeting variable? We're already getting some pressure on how much data is going to the ad server.

@mkendall07 https://github.com/mkendall07 - making these fields mandatory or strongly encouraged seems like a candidate for 4.0?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/prebid/Prebid.js/issues/3115?email_source=notifications&email_token=AAM25ZZ3OHHKVJC7MWPTKGTQQZKHHA5CNFSM4FWRPVQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECLR7JA#issuecomment-546774948, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM25Z4RSQPWT72BL6RO4E3QQZKHHANCNFSM4FWRPVQQ .

bretg commented 4 years ago

I took a first step and opened a docs PR to at least document the meta object. Made some changes to the proposal:

changed clickUrl to 'advertiserDomains'. Taking into account other advice, the domain of the clickUrl is not always the final landing page after a series of redirects. I used 'advertiserDomain' because it aligns with the OpenRTB 'adomain' field, which is an array in case this is a 'rotating creative'.
changed buyerId and buyerName to advertiserId and advertiserName to align with advertiserDomain. Also, 3 adapters already support advertiserId. Its a start.

I can be talked out of these changes.

@Slind14 and @patmmccann - need your review and suggestion about which fields are the most critical and should ideally be made "mandatory" in some future release.

Will leave the next step to @mkendall07. One suggestion would be to reach out to all 200+ adapters and warn that the meta object may someday be required, and in any case, is desired by publishers.

Slind14 commented 4 years ago

I'm good with these adjustments. If it is possible to send out an email to all the maintainers of the adapters, that would be fantastic.

The most important ones are network/dsp and buyer/advertiser followed by brand.

bretg commented 4 years ago

Excuse the likely ignorance here, but what number/namespace should be reported? i.e. is there a standard numbering schema/naming system?

advertiserDomains seems to be the best bet on that side.
if there isn't another, maybe GDPR vendor ID could be a standard there?

bretg commented 4 years ago

Before making a big deal of this to the Prebid bidders, I'd like to have a stronger case. It's arguable that the only really useful fields are Advertiser Domain and IAB categories.

The lack of a global ID/name space for companies seems to eliminate most of the value in having that metadata. The major exchanges noted above are reporting their own internal IDs, which doesn't seem very useful for pubs who want to implement reporting or automated blocking.

Or would you argue that even bidder-specific IDs are better than none because at least blocking is possible?

sdwhite89 commented 4 years ago

Is your argument in reference to existing fields or new fields? I'd like to echo what nils has suggested, how that would look, I'm not sure, but those fields are super important from a publishers perspective.

Cheers,

On Tue, 7 Apr 2020 at 15:24, bretg notifications@github.com wrote:

Before making a big deal of this to the Prebid bidders, I'd like to have a stronger case. It's arguable that the only really useful fields are Advertiser Domain and IAB categories.

The lack of a global ID/name space for companies seems to eliminate most of the value in having that metadata. The major exchanges noted above are reporting their own internal IDs, which doesn't seem very useful for pubs who want to implement reporting or automated blocking.

Or would you argue that even bidder-specific IDs are better than none because at least blocking is possible?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/prebid/Prebid.js/issues/3115#issuecomment-610417065, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALC6LE3KZXONQBVV2G7RD2LRLMZQ7ANCNFSM4FWRPVQQ .

--

Spencer White Programmatic Platforms and Yield Manager Guardian News & Media

(+44) 020 335 33410 spencer.white@guardian.co.uk

Kings Place, 90 York Way, https://maps.google.com/?q=90+York+Way,%C2%A0+London+N1+9GU&entry=gmail&source=g

London N1 9GU https://maps.google.com/?q=90+York+Way,%C2%A0+London+N1+9GU&entry=gmail&source=g

theguardian.com

--

This e-mail and all attachments are confidential and may also be privileged. If you are not the named recipient, please notify the sender and delete the e-mail and all attachments immediately. Do not disclose the contents to another person. You may not use the information for any purpose, or store, or copy, it in any way. Guardian News & Media Limited is not liable for any computer viruses or other material transmitted with or as part of this e-mail. You should employ virus checking software.

Guardian News & Media Limited is a member of Guardian Media Group plc. Registered Office: PO Box 68164, Kings Place, 90 York Way, London, N1P 2AP. Registered in England Number 908396

Slind14 commented 4 years ago

It works incredible well for automatically blocking bad creatives (mobile redirects for example).

Furthermore it is not that difficult to normalize it.

And I'm sure it is more likely for publishers to collaborate on a public mapping of all the different exchange id's than it is for exchanges to implement a standard (same ids).

bretg commented 4 years ago

automatically blocking bad creatives

That's the purpose of the 'creativeId' field, not agencyId or networkId.

Blocking on advertiser and category make sense.

those fields are super important from a publishers perspective

I partly understand why this data might be interesting (reporting and blocking), but I'm skeptical it will scale across many bidders. Here's the vision I've gathered so far:

all 300 bidders should update their adapters to provide bidder-specific advertiser/agency/network IDs and names. The IDs may not be useful to most publishers. Names might vary in detail. e.g. variations on "The TradeDesk", "TTD", "trade desk", "tradedesk, inc", etc.
interested publishers willing to put the effort in will come up with regular expressions to normalize messy variations

Furthermore it is not that difficult to normalize it.

Maybe not difficult for the important agencies or a few advertisers, but this problem is hard at scale - thousands of advertisers tracked across name-changes, acquisitions, and typos. Theoretically someone with enough energy could build/maintain a Prebid.js module that would contain all the needed regex or mapping files, but it would probably kill the average browser. This problem is more obviously solved in Prebid Server.

Anyhow, you guys live this -- please help us by suggesting concrete reasons why bidders should spend time doing this. It will help build a compelling case that's more likely to prompt action.

1) publishers want to know which advertisers and brands are being delivered into their pages so they can trace and block ad content that may not fit into their context 2) publishers want to know which ad networks and agencies are delivering to their pages so they can trace and block entities that consistently serve bad creatives (?) 3) ...

Slind14 commented 4 years ago

That's the purpose of the 'creativeId' field, not agencyId or networkId.

The creative id is useless for this, there are often hundreds of them for the same malware campaign. Most of the times there are also multiple brands and sometimes multiple advertisers. They can also take any path they want.

So we start at the top with the exchange then going down to DSP, advertiser and finally brand. Trying to block the smallest possible one.

We also want this information to block them in the exchanges settings, so they won't lock up the exchange when we are throwing the bid away.

all 300 bidders should update their adapters to provide bidder-specific advertiser/agency/network IDs and names. The IDs may not be useful to most publishers. Names might vary in detail. e.g. variations on "The TradeDesk", "TTD", "trade desk", "tradedesk, inc", etc.

Id's are needed because they won't change. The names can change at any time which would break the publishers normalization.

interested publishers willing to put the effort in will come up with regular expressions to normalize messy variations

In most cases it is enough to normalize the top 200 but we still want the id and name for the identifying who bought it when something goes wrong like malware or performance intense creatives.

Maybe not difficult for the important agencies or a few advertisers, but this problem is hard at scale - thousands of advertisers tracked across name-changes, acquisitions, and typos. Theoretically someone with enough energy could build/maintain a Prebid.js module that would contain all the needed regex or mapping files, but it would probably kill the average browser. This problem is more obviously solved in Prebid Server.

This happens server-side in the publishers reporting. There is no need to do any of that in prebid.

In our case we even went so far that we stopped working with all the exchanges which do not provide at least one of these dimensions. This shows how important it is to us.

bretg commented 4 years ago

ok, please take a look at https://prebid.org/dev-docs/bidder-adaptor.html and search for meta. Not much but it's a start. Should IAB categories be strings prefixed with "IAB-" or just integers?

Slind14 commented 4 years ago

Looks good. Thank you.

bretg commented 4 years ago

Also updated the server-side bidder adapter page -- http://lh.prebid.org:8080/prebid-server/developers/add-new-bidder.html#bid-response-metadata

bretg commented 3 years ago

@Slind14 - we'd like to propose closing out this issue:

1) The PBJS bidder documentation encourages adapters to define these values on the bidresponse https://docs.prebid.org/dev-docs/bidder-adaptor.html 2) Same with the PBS bidder docs - https://docs.prebid.org/prebid-server/developers/add-new-bidder-go.html 3) @gglas is going to open a separate issue to allow publishers to configure whether a certain piece of metadata is required for accepting bid responses.

Do you agree this can be closed, working the enforcement in a separate issue?

Slind14 commented 3 years ago

Thank you, @bretg

Is there any chance that it can be made mandatory with the next breaking release?

bretg commented 3 years ago

@Slind14 - we're going to make it optional for publishers to configure whether they want to enforce certain fields for certain bidders. Prebid is not in a position to push the entire industry to supply metadata.

It's conceivable that such opt-in behavior could become default in the distant future.

ChrisHuie commented 3 years ago

closing and moving to #6453

prebid / Prebid.js