Modeling Confidence in XDM Data

[trieloff]

Based on a conversation with @nangia: a key aspect of digital marketing is that the identity of the consumer is largely probabilistic. Whether products make it explicit or not, the data that we have about a consumer is messy, and largely untrusted. A profile is typically constructed from multiple data sources, including:

• direct user input (although users regularly enter false information in online forms)
• inferred from user behavior (with implicit inaccuracy)
• acquired from third-party data sources (with implicit inaccuracy)
• through rule-based or probabilistic merging of profiles (with explicit inaccuracy)

Therefore, a system that represents consumer profiles in a digital marketing world needs to be able to express the inaccuracies, eventual contradictions, and missing data that occurs in the real world. This is a key differentiator from the way how collaboration systems or CRM systems look at the world (they have an inherent assumption of clean data).

We need a way to express these scenarios in JSON.

What are the schemas that are affected by the issue

• Profile

What are examples of products that are impacted by the issue

• Adobe Target
• Adobe Analytics
• Adobe Audience Manager
• Adobe Media Optimizer
• Adobe Social

[cdegroot-adobe]

We need to have a more specific driving use case rooted in practical terms before we invest in designing this. Many aspects of non-deterministic identity resolution are at the implementation and usage context level. So may not appear in XDM, but is data usage patterns.

[cdegroot-adobe]

@trieloff - I propose we close this and track the requirement as teams come on board that have this requirement.

[trieloff]

I disagree. We have #247, which is an open issue and product-related. In addition, we do have a basic model of probability in the schemas already with machineKeywords.

A few things that we need to model:

• probability of an event occurring
• probability of a metric reaching a threshold
• probability of a fact being true

Each of these needs

• a timespan for which the prediction is valid
• the information who has made the prediction and when
• the ability to hold multiple predictions in a list