Plugin System for Prebid

CJxD commented 3 years ago

Background

NumberEight has an SDK which provides first-party targeting options for unauthenticated users by reading sensor data from the user's phone. We'd like to offer publishers the ability to add NumberEight's signals to Prebid requests in a seamless and faff-less experience.

Today, publishers can retrieve targetable audience IDs directly from the app which are then inserted in bid streams. An example may be:

func getAudiences() {
    let audiences = Audiences.currentMemberships
    for membership in audiences {
        NSLog("Audiences \(membership.id)") // e.g. NE-2-11
        NSLog("Audiences \(membership.name)") // e.g. Coffee Lovers
    }
}

Proposed Solution

Overview

Thinking broadly, beyond just integration with NumberEight, it makes sense to have a third-party plugin system for Prebid. This will initially be for additional targeting parameters in OpenRTB format, but further plugin functionality can be added to extend this concept.

The experience of the publisher would be as follows:

let nePlugin = NumberEight.prebidPlugin
Prebid.registerPlugin(plugin: nePlugin)

Class Diagram

Class names are taken from the Android copy of Prebid.

Prebid FPD Targeting Params Class Diagram v2 vpd

Sequence Diagram

Class names are taken from the Android copy of Prebid.

Prebid FPD Targeting Params Sequence Diagram v2 vpd

Details

The PrebidPlugin abstract class is extended by the plugin developer. The only method today is getTargetingParamsProvider() which allows the plugin to provide app and user objects for OpenRTB requests (see the Seller Defined Audiences proposal). By default, such methods return null, i.e. no implementation.
The TargetingParamsProvider class is invoked every time a bid request is made by the PrebidServer. The PrebidServer is modified to request entire OpenRTB objects from the TargetingParams class via getUserObject() and getAppObject(), rather than individual fields like age and gender today.
The TargetingParams class is responsible for merging together the RTB objects provided by the plugins with its own (as produced by the private functions paramsToUserObject() and paramsToAppObject()). It merges the objects using the OpenRtbUtils helper class.
The TargetingParamsProvider includes a method updateAvailable() to indicate that data has changed since the last request for OpenRTB data, such that the TargetingParams class can cache pre-merged OpenRTB objects to save time. This is stored in userObjectCache and appObjectCache.

Unresolved questions

Should targeting parameters be limited to only the app and user objects of OpenRTB?
What keys should be allowed or disallowed in the lintOpenRtbObject(object, keys) process?
Who gets preference when two plugins provide the same keys in an OpenRTB object?

YuriyVelichkoPI commented 2 years ago

Hi @CJxD ! I'm working on the general proposal for the SDK plugins based on this request. I'll try to finalize the draft version by the next PMC.

So far I have some questions about your diagrams.

According to the diagrams you propose to get the whole App and User objects from the plugins. However, the "Taxonomy and Data Transparency Standards to Support Seller-defined Audience and Context Signaling" says that User.data and app.content.data should be used for taxonomy signals. So I'm wondering which other user and app fields should be customizable for your purposes. And how do you see the conflict-solving policy for publishers if more than one provider needs to modify single-value fields in these objects?

For now, I think that we can consider, for example, prioritization of plugins on registering for single-value properties. But need to think more about this.

CJxD commented 2 years ago

Hey @YuriyVelichkoPI!

My thoughts behind exposing the whole App and User objects (and perhaps more) is with the forethought that perhaps there will be plugins that automatically manage data beyond just seller-defined audiences, such as app.content, app.keywords, user.gender, user.geo, etc. It's more of a future-play to encourage innovations that augment the entire OpenRTB request.

Of course, the point about conflict resolution is then quite important, and hence the mergeOpenRtbObjects() function. There are several strategies that could be used, from giving preference to the plugin registered last, or having a priority integer in the registration of the plugin (e.g. TargetingParams.registerProvider(provider, 100)), similarly to how DNS MX records have priorities.

Thoughts?

On Mon, 17 Jan 2022 at 11:02, Yuriy Velichko @.***> wrote:

Hi @CJxD https://github.com/CJxD ! I'm working on the general proposal for the SDK plugins based on this request. I'll try to finalize the draft version by the next PMC.

So far I have some questions about your diagrams.

According to the diagrams you propose to get the whole App and User objects from the plugins. However, the "Taxonomy and Data Transparency Standards to Support Seller-defined Audience and Context Signaling" says that User.data and app.content.data should be used for taxonomy signals. So I'm wondering which other user and app fields should be customizable for your purposes. And how do you see the conflict-solving policy for publishers if more than one provider needs to modify single-value fields in these objects?

For now, I think that we can consider, for example, prioritization of plugins on registering for single-value properties. But need to think more about this.

— Reply to this email directly, view it on GitHub https://github.com/prebid/prebid-mobile-ios/issues/461#issuecomment-1014393953, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMN37EGFYWE4EGHKK2JGALUWPZLRANCNFSM5JHTSBJQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

YuriyVelichkoPI commented 2 years ago

In general, I think in the same way.

But I'd like to prevent potential fraud and content manipulation by different plugins. For instance, I believe that properties like app.name or app.bundle, should be set explicitly by publishers or even by SDK but not by third-party vendors.

If to talk about the User object, yes it looks like a third-party vendor can manage all fields.

However, I'd consider an "add" approach instead of "merge" one. And provide separated methods for each user property. For instance getUserYob(), getUserData(). It should allow assembling the object in a more clear and reliable way. Does it make sense?

If to talk about conflicts, the prioritization or sequential order looks fine until the publisher wants to prioritize PuginA for the PropertyX and PluginB for PropertyY. Maybe we can handle it with prioritization by categories, like register(plugin: plugin, priorForCategories: [UserYob, Contenttaxonomy]). But still encourage publishers to use such kind of registration only in edge cases. In most cases, the prioritization by registration order should work.

CJxD commented 2 years ago

I still can't reply directly through GitHub for some reason (maybe I don't have permission to comment?), so replying through email again. Excuse the markdown!

You're certainly right about restricting particular keys like app.name, that's the job of lintOpenRtbObject() in this case: blacklisting (or whitelisting) properties of the objects that may be set by third parties.

I think in the long run, manipulating JSON objects will be easier to maintain than having individual getter/setter methods for fields of the OpenRTB object e.g. getUserYob() - it avoids the need to change the interface every time there is an update to the OpenRTB specificiation.

Giving priority for keys as you've mentioned would work well. I would suggest that the keys use object dot notation so that they match the OpenRTB spec, such as TargetingParams.register(provider: provider, priorities: ['user.yob']).

For the top level PrebidMobile.register(plugin: plugin) method, since there may be plugins other than targeting params providers, there should be an options struct containing the above, something like this:

let opts = PrebidMobilePluginOptions()
opts.targeting.priorities = ['user.yob']
PrebidMobile.register(plugin: plugin, options: opts)

or even a dict:

PrebidMobile.register(plugin: plugin, options: [
    .targetingPriorities: ['user.yob']
])

On Thu, 20 Jan 2022 at 09:02, Yuriy Velichko @.***> wrote:

In general, I think in the same way.

But I'd like to prevent potential fraud and content manipulation by different plugins. For instance, I believe that properties like app.name or app.bundle, should be set explicitly by publishers or even by SDK but not by third-party vendors.

If to talk about the User object, yes it looks like a third-party vendor can manage all fields.

However, I'd consider an "add" approach instead of "merge" one. And provide separated methods for each user property. For instance getUserYob(), getUserData(). It should allow assembling the object in a more clear and reliable way. Does it make sense?

If to talk about conflicts, the prioritization or sequential order looks fine until the publisher wants to prioritize PuginA for the PropertyX and PluginB for PropertyY. Maybe we can handle it with prioritization by categories, like register(plugin: plugin, priorForCategories: [UserYob, Contenttaxonomy]). But still encourage publishers to use such kind of registration only in edge cases. In most cases, the prioritization by registration order should work.

— Reply to this email directly, view it on GitHub https://github.com/prebid/prebid-mobile-ios/issues/461#issuecomment-1017254559, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMN37AIJLD3FQOUE5N5MQLUW7FTJANCNFSM5JHTSBJQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

YuriyVelichkoPI commented 2 years ago

hey, @CJxD, I've added you to the repo members, so you should have the ability to comment now.

I like the idea of black-listed properties on merging. However, not sure about the benefits of JSON approach. I'd prefer to get data in predefined structs, because, changes in the JSON on the third-party side can lead to unexpected runtime behavior. I don't say about crashes but the validation method can just fail silently somewhere in prod and as the result, some data will be missed in the request. On the other hand, if we provide typed API, like classes App, Data, User. and others it will force vendors to provide data in the "recognizable" way from the very begging. And such an approach will allow avoiding parsing issues.

YuriyVelichkoPI commented 2 years ago

I've been starting preparation of the implementation doc: https://docs.google.com/document/d/1t6J7lBHsLvKvKn7J92BgTIj-CG4WJ3Luh-FnsOI5s1k/edit#heading=h.94gaf9433upe

I'll move all agreements into it to get a transparent single source of truth in the end.

CJxD commented 2 years ago

Looks good Yuriy :)

CJxD commented 2 years ago

I've just seen that Prebid Mobile does actually have a modules system as in Prebid.js https://docs.prebid.org/prebid-mobile/modules/modules-overview.html https://docs.prebid.org/dev-docs/modules/

This is absolutely what the spirit of this proposal is, so any development to this extent should just be making it easier for modules to interact with bids.

As such, the wording 'plugin' should be replaced with 'module', e.g. Prebid.registerModule(module: module)

YuriyVelichkoPI commented 2 years ago

YuriyVelichkoPI commented 6 months ago

This feature is not completed, and it is still actual.

prebid / prebid-mobile-ios