PBS GPP phase 3

[bretg]

Splitting up the GPP feature tracked originally by #2380.

Phase 3 is the "Activity Infrastructure". From the document, this phase is described as:

Prebid.js and Prebid Server will define a set of special activities, each of which can be controlled either directly by the publisher or, in the future, via consent modules like GPP. The activities defined for Prebid Server are in the document section 3.3.4.

The work for this phase includes:

1. Support a "module tags" field that can be applied to the module level. This will be used to tag the module with a type, e.g. "rtd" or "userid". This is the component_type field. "General" is the default module type.
2. Implement the call to each Activity. The interface will need to be specified. It might make sense to do some performance testing for whether activity interfaces should be called once-per-bidder/module or just once and return a list of allowed/disallowed bidders/modules.
3. For the /auction endpoint, implement the processing of the account config per the publisher's specifications, including support for rules, componentName, and componentType.
4. The /cookie_sync and /setuid endpoints need to be able to read account-level config for the syncUser activity.
5. There should be adequate debug tracing. ext.prebid.trace: "basic", "verbose"
6. There should also be adequate metrics. See the metrics section below.

Details

Account-level config defines an ordered set of rules by which the system determines whether a given activity is allowed.

Here's the general form:

allowactivities: {
        ACTIVITY: {
            default: true/false,
            rules: [{
              condition: { CONDITION_LOGIC },
              allow: true/false
            },{
              futureRuleAttribute: futureRuleValue
            },{
              ... more rules ...
            }]
        }
    }

Definitions:

• default: defines whether to perform this activity if no overrides or rules fire. If default is unspecified, the hardcoded default-default is true.
• rules: defines zero or more conditions. Rules are processed in array order for their true/false outcomes. Rules processing stops after the first rule produces a result.

Note that rules can have no conditions, in which case they automatically match and the "allow" value is utilized.

Conditions

In general, a condition is a series of clauses that are processed separately. Only if all clauses present are true is the entire condition true.

condition: {
    CLAUSE1,
    CLAUSE2,
    ...
}

The general forms a condition take:

condition: {
    ATTRIBUTE1: [array of strings],
    ATTRIBUTE2: [array of strings]
}

Notes:

• array-of-string is the only datatype supported initially, but additional types may be added in the future
• wildcarded strings may be added in the future
• if the datatype of the attribute is not an array of strings, that rule is invalid. An empty array is invalid as well.
• If a rule is invalid for any reason, skip it and emit an error at N% sampling.

Requirements:

1. CLAUSES are ANDed together. i.e. the result of every ATTRIBUTE clause must be true for the entire condition to be true.
2. It's undefined what happens when a given ATTRIBUTE is present in the condition more than once.

The following attributes may be seen:

1. componentName - this is the biddercode, the analytics adapter code, or the module code.
2. componentType - this is "bidder", "analytics", or for modules: if the module tag contains "rtd", then type is "rtd", otherwise type is "general".

Metrics

Metrics will track the number of disallowed activities:

• requests.activity.ACTIVITY.disallowed.count
• accounts.ACCOUNT.activity.ACTIVITY.disallowed.count - at 'verbose' metric level
• adapter.ADAPTER.activity.ACTIVITY.disallowed.count

An additional metric will track the overall number of rules processed across activities:

• requests.activity.processedrules.count
• accounts.ACCOUNT.activity.processedrules.count - at 'verbose' metric level

Module Stages

Every module stage is subject to the transmit activities except the entrypoint stage. This is because the account ID is not yet resolved, so account config is not yet available. This means that modules employing the entrypoint stage must not utilize user-specific data (UFPD, EIDS, IP address).

Example 1

Account-level activity controls:

{
  privacy: {
    allowactivities: {
      transmitEids: { // ok for everyone except bidderA
        default: true,
        rules: [{
          condition: {
            componentName: ["bidderA"]
          },
          allow: false
        }]
      },
      transmitUfpd: {
        default: false,  // let only certain modules get UFPD
        rules: [{
          condition: {
            componentType: ["bidder", "analytics"]
          },
          allow: false
        },{
          privacyRegs: ["*"]    // future link to GPP/etc. Syntax may change.
        }]
      },
      ...
    }
  }
}

Example 2

Let GPP decide whether to transmitUfpd, but by default let RTD and Analytics modules do it.

{
  privacy: {
    allowactivities: {
      transmitUfpd: {
        default: false,
        rules: [{
            privacyRegs: ["*"]     // syntax may change as we develop GPP phase 4
          },{
          condition: {
            componentType: ["rtd", "analytics"]
          },
          allow: true
        }]
      }
    }
  }
}

[bretg]

Discussed in PBS committee. Giving time to committee members to review in detail.

[bretg]

Spoke with the PBJS committee -- there's some possible changes in the syntax coming. Moving back to "needs req" while that discussions settles.

[bretg]

Ok, updated with the current state of the Prebid.js discussion.

[dgirardi]

default: defines whether to perform this activity if no overrides or rules fire. If default is unspecified, the hardcoded default-default is false.

I read that to mean "everything is denied unless explicitly allowed". Is that true? in JS, the default-default is true (everything is allowed unless explicitly denied).

It's ok for the same ATTRIBUTE to be present in the condition more than once.

If attributes are map keys, they can't really be present more than once. They can be present in different conditions from different rules.

sync_method - PBS should ignore and resolve this clause to true. This is a Prebid.js requires this logic, but PBS does not.

The purpose of this is for JS to say things like "allow image but not iframe user syncs". If that's a distinction that doesn't make sense server-side, that tells me that it's not a question of privacy control, and we may want to drop it from JS as well (reduce the control to "allow/disallow user syncs", and keep the iframe/image toggle configuration somewhere else).

[bretg]

in JS, the default-default is true

Updated here.

say things like "allow image but not iframe user syncs". If that's a distinction that doesn't make sense server-side, that tells me that it's not a question of privacy control, and we may want to drop it from JS as well

This is worth bringing up in the PBJS committee.

Option 1) integrate the fine-grained behavior of iframe/image usersync into the activity infrastructure right now. Option 2) just worry about yay/nay for now, leaving the iframe/image details alone for now

Specifically for Prebid Server, this is the /cookie_sync endpoint, not the /auction endpoint. /cookie_sync doesn't support ORTB requests -- it's a POST body with a few parameters, which does include the iframe/image details.

My take: I think this project is big enough as-is. Leave the iframe/image sync details out of the activity infra for now.

[bretg]

Removed these from scope:

• Modules must also support the ability to be associated with a unique Prebid-Server 'component' namespace and one or more vendor IDs, e.g. GVL ID.
• Analytics adapters should support the ability to be associated with a unique component namespace and one or more vendor IDs, e.g. GVL ID.

Updated the condition logic.

[bretg]

Removed the request-level activity controls.

[bretg]

Discussed with the PBS GPP sub-committee. We've decided to go back to the original "script-based" approach for these reasons:

• there's no need for explicit priority. Instead, priority is implicit in the ordering of the rules
• this can help server-side performance in a potentially significant way because processing can stop once the first rule matches.

[dgirardi]

there's no need for explicit priority. Instead, priority is implicit in the ordering of the rules

this only works if all the rules are defined in one place. the reason we made it a requirement is so that the pub would say "here's my rules, but also use your own GPP rules". Priority was a mechanism to tell whether the pub's rule would override other rules.

[SyntaxNode]

Priority was a mechanism to tell whether the pub's rule would override [GPP] rules.

The priority mechanism was not removed. Priority is now expressed as the ordinal position in a JSON array "script" rather than an explicit number. The concept of "use your own GPP rules" is represented as a call out to the privacy policies. This is the original approach for PBS before we begun to align with PBJS. We would like to revert to this original approach as it's been mutually determined that alignment with PBS and PBJS on this syntax is not necessary (we'll continue to align on privacy activity names and definitions).

The examples will be updated. The approximate new syntax is:

{
  consent: {
    allowactivities: {
      transmitEids: { // ok for everyone except bidderA
        default: true,
        rules: [{
          condition: {
            componentName: ["bidderA"]
          },
          allow: false
        },
        {
          callPrivacy: ["*"]
        }]
      },
      . . .
    }
  }
}

The callouts to privacy policies/regulations can be ordered in any desired arrangement and can be explicitly broken apart by using a policy identifier (tcfeuv2) in place of the catch-all (*).

[bretg]

@dgirardi - the server development team pushed the implementation back to the "centralized script" model. My intuition from January seems to have been validated, and now that client- and server-side coordination is no longer needed, I don't see a need to hold them to the more expensive rules-based system.

I suspect you're likely to object that the rules-based system isn't more expensive, but the server engineers disagree: processing can more easily be stopped short when publisher config is going to completely obviate anything the regulations might say.

[dgirardi]

Very well - just wanted to make that clear since I know it was a point of confusion during design; I think I hear that you do, in fact, have all the rules in one place now, so priority is not needed. I'll keep my complaints about that for other occasions :)

[bretg]

Please note that the main GPP doc has been updated for Prebid Server to combine the transmitUfpd and transmitEids activities. In the future we may break them out if necessary.

[bretg]

Updated to move the account config under privacy rather than consent.

[bretg]

Done in PBS-Java