ADR: obligations implementation path

[jakedoublev]

Table of Contents

1. Background
2. Option 1: Obligations as a separate policy construct
3. Option 2: Obligations via flag within existing attributes as previously detailed
4. Level of Effort
5. Pros/Cons

Background

Relevant ADR: https://github.com/opentdf/platform/issues/874 Relevant implementation issue: https://github.com/opentdf/platform/issues/1260

Example PEP obligations once Subject entity found to be entitled in a data access decision:

• watermark
• change-all-images-to-ducks
• make-font-size-20

Option 1: Obligations as a separate policy construct

• separate FQN for obligations https://<namespace>/oblg/<name>
• no parent/child relationship among obligations (they're flat and distinct from one another) but nothing stops you from naming it https://namespace/oblg/drm:watermark
• no rule like attributes, or implicit anyOf rule like we've discussed

erDiagram
    obligations {
        uuid id
        uuid namespace_id FK
        string name
        jsonb feature_context "demonstration: we could add robust feature context to an obligation like an expiration timestamp in whatever new columns"
        jsonb metadata "conventional policy metadata labels key/value store"
        compIdx      comp_key     UK "namespace_id + name"
    }

    obligation_assignments {
        uuid id
        uuid obligation_id
        uuid attribute_value_id
    }

    obligation_fulfillments {
        uuid id
        uuid obligation_id
        enum entity_scope "subject | environment"
        jsonb fulfillment_conditions "condition sets not tied to subjects"
        jsonb metadata "conventional policy metadata labels key/value store"
    }

    attribute_values {
        uuid         id                      PK "abbreviated table"
        uuid         attribute_definition_id FK
        varchar      value
    }

    namespaces {
        uuid        id   PK
        varchar     name UK
    }

    attribute_definitions {
        uuid         id           PK "abbreviated table"
        uuid         namespace_id FK
        varchar      name
        enum         rule
    }

    namespaces ||--|{ attribute_definitions : has
    namespaces ||--|{ obligations : has
    attribute_definitions ||--|{ attribute_values : has
    obligations ||--o{ obligation_assignments : links
    attribute_values ||--o{ obligation_assignments : links
    obligations ||--o{ obligation_fulfillments : "fulfilled by"

    %% abbreviations
    subject_mappings ||--o{ attribute_values : entitles
    subject_condition_sets ||--o{ subject_mappings : relates
        kas_val_grants ||--o{ attribute_values : grants
    kas_attr_grants ||--o{ attribute_definitions : grants

Policy work:

1. new obligations table
2. new obligation_assignments table linking derived obligations and attribute_values
3. new obligation_fulfillments table
4. obligation fulfillment conditions protos (not purely subject-oriented, so can't reuse SCS from entitlements/SubjectMappings without breaking changes)
5. CRUD RPCs and db functions for:
   1. CRUD obligations
   2. assign/unassign obligations to attribute values
   3. CRUD obligation_fulfillments for obligations
6. tests for db CRUD of obligations, assignments, fulfillments
7. docs for CRUD of obligations, assignments, fulfillments
8. FQN lookup for obligation values (reuses existing FQNs table?) https://namespace/oblg/<name>, i.e. https://demo.com/oblg/drm:watermark or demo.com/oblg/readonly
9. obligations RPCs/service rolled up into policy with dedicated casbin policy
10. new CLI CRUD support for obligations:
    1. CRUD obligations
    2. assign/unassign obligations to attribute values
    3. CRUD obligation_fulfillments for obligations
11. Value protos are updated to include derived obligations
12. GetAttributesByValueFQNs adds support in protos/db to return derived obligations for values
13. GetObligationsByFQNs (new RPC) expects obligation FQN structure and returns obligation fulfillment conditions

Authz/KAS/SDK work:

1. Read derived obligations from resource attributes
2. Contextualize obligations on TDF data policy
3. Logic/tests to resolve obligation condition logic
4. SDK must allow obligation-FQN-structure strings alongside data attribute FQN strings
5. TODO: AccessPDP resolves obligation fulfillments as scoped (multiple obligations/multiple entities = what behavior?)
6. GetDecisions calls AccessPDP correctly and returns obligations correctly
7. KAS returns obligations along with /rewrap
8. JS SDK support for obligations in response?
9. Go SDK support for obligations in response?

Option 2: Obligations via flag within existing attributes as previously detailed

erDiagram

    obligation_assignments {
        uuid id
        uuid obligation_attribute_value_id
        uuid data_attribute_value_id
    }

    obligation_fulfillments {
        uuid id
        uuid obligation_attribute_value_id
        enum entity_scope "subject | environment"
        jsonb fulfillment_conditions "condition sets not tied to subjects"
        jsonb metadata "conventional policy metadata labels key/value store"
    }

    attribute_values {
        uuid         id                      PK
        uuid         attribute_definition_id FK
        varchar      value
        jsonb        metadata
        compIdx      comp_key                UK "ns_id + ad_id + value"
    }

    namespaces {
        uuid        id   PK
        varchar     name UK
    }

    attribute_definitions {
        uuid         id           PK
        uuid         namespace_id FK
        varchar      name
        enum        type "data | obligation"
        enum         rule "enforced by policy services to be anyOf if obligation type"
        jsonb        metadata
        compIdx      comp_key     UK "ns_id + name"
    }

    namespaces ||--|{ attribute_definitions : has
    attribute_definitions ||--|{ attribute_values : has
    attribute_values ||--o{ obligation_assignments : "data attr val -> 1+ obligations"
    obligation_assignments ||--o{ attribute_values : "obligation -> 1+ data attr vals"
    attribute_values ||--o{ obligation_fulfillments : "IFF obligation, fulfills"

    %% abbreviations
    subject_mappings ||--o{ attribute_values : "IFF data attr def val, entitles"
    subject_condition_sets ||--o{ subject_mappings : relates
    kas_val_grants ||--o{ attribute_values : "IFF data attr def val, grants"
    kas_attr_grants ||--o{ attribute_definitions : "IFF data attr def val, grants"

Policy work

1. policy database migration to:

   1. add type ENUM('data','obligation')
   2. write all current platform attributes to the 'data' type

2. new obligation_assignments table linking derived obligation_attribute_values and data_attribute_values

3. new obligation_fulfillments table

4. Update attributes CRUD to support obligations via type enum flag

   1. Create
      1. attributes are implicitly defaulted 'data' type unless defined as 'obligation'
      2. accept type on Create in attr definition protos/rpcs
      3. implicitly default anyOf rule only on 'obligation' type Create, or decide to enforce 'anyOf' and fail?
      4. Update proto comments about two types of attributes and what they mean
      5. tests + docs
   2. GET/LIST
      1. Update Value to query derived obligations from DB and set in response IFF 'data' attribute
      2. Update Value proto to provide obligation_fulfillments IFF 'obligation' attribute
      3. Update Definition proto to provide obligation type in response
      4. tests + docs
   3. UNSAFE
      1. enforce cannot change rule on an obligation-type attribute (unless we want to support rules in obligations other than anyOf?)
      2. tests + docs

5. Update other policy CRUD to differentiate between attribute types

   1. subject mappings
      1. Create must fail if mapped to an obligation-type attribute value
      2. Update must fail if mapped to an obligation-type attribute value
      3. tests + docs
   2. resource mappings
      1. Create must fail if mapped to an obligation-type attribute value
      2. Update must fail if mapped to an obligation-type attribute value
      3. tests + docs
   3. KAS grants to attribute definitions
      1. Create must fail if mapped to an obligation-type attribute value
      2. Update must fail if mapped to an obligation-type attribute value
      3. tests + docs
   4. KAS grants to attribute values
      1. Create must fail if mapped to an obligation-type attribute value
      2. Update must fail if mapped to an obligation-type attribute value
      3. tests + docs

6. obligation fulfillment conditions protos (not purely subject-oriented, so can't reuse subject condition sets without breaking changes)

7. FQN lookup for obligation values (no change) https://namespace/attr/obligation_name/value/<value>, i.e. https://demo.com/attr/drm/value/watermark

8. CLI CRUD for obligations (still separate subcommands to improve documentation specificity and clarity)

   1. CRUD obligations via policy attribute RPCs under the hood
   2. assign/unassign obligations to attribute values
   3. CRUD obligation_fulfillments to obligation_values

9. Hide obligation type attributes/values from existing CLI CRUD of those policy objects

10. docs/tests for new CRUD behaviors (assignments, fulfillments)

11. Value protos are updated to include derived obligations

12. GetAttributesByValueFQNs adds support in protos/db to return derived obligations for values

13. GetAttributesByValueFQNs is updated to have a different response if the requested FQN was an obligation-type attribute value

Authz/KAS/SDK work:

1. Read derived obligations from resource attributes
2. Contextualize obligations on TDF data policy
3. Logic/tests to resolve obligation condition logic
4. TODO: AccessPDP resolves fulfillment conditions by scope (multiple obligations/multiple entities = what behavior?)
5. GetDecisions calls AccessPDP correctly and returns obligations correctly
6. KAS returns obligations along with /rewrap
7. JS SDK support for obligations in response?
8. Go SDK support for obligations in response?

Level of Effort (LOE) cost comparison to implement, maintain, understand, and develop

Path 1: Distinctly Separate Obligation Constructs	Work	LOE
	Initially implement	M/L
	Maintain as obligations and attributes each evolve	S/M
	Understand for admins	S/M
	Understand as developers building on top	S/M

Path 2: Obligations as Attribute Types	Work	LOE
	Initially implement	M/L
	Maintain as obligations evolve (unknown but coupled with shared concerns)	M/L
	Understand for admins	M
	Understand as developers building on top	M

Other data points

1. Adding the unsafe policy service was 9 new RPC endpoints, db functionalities with tests, 9 new CLI paths, and docs and took one engineer (myself) ~2 weeks with around 2/3 focus
2. We don't know exactly how obligations will evolve/mature over time (DLP, malware scanning, watermark)
3. Making obligations attributes means they are parent/child, rule-associated (implicitly), with an FQN that is ambiguous as to attribute type

4. The two options mostly overlap in cost to implement beyond policy, with the following roughly the same:

   1. KAS/Authz/SDK/AccessPDP support for obligations work once read from policy services
   2. CLI support for obligations work

Pros/Cons of Options

Option 1: Obligations as a separate construct

• :green_square: Good, because long-term costs are expected to be lower (maintainance, testing, onboarding)
• :green_square: Good, because more flexibility evolving obligations apart from attributes as our understanding of use cases matures
• :green_square: Good, because less implicit behavior (requiring anyOf rule and parent/child structure)
• :green_square: Good, because we can unblock differentiated RBAC casbin auth on obligations distinct from attributes
• :green_square: Good, because it's less burden on admins to understand, and easier to understand when totally separate from attributes (i.e. "here is what an attribute means, and here is what an obligation means")
• :green_square: Good, because smaller blast radius for bugs and breaking changes
• :green_square: Good, because nothing stopping us from eventually deriving obligations off entities with separate constructs used while deriving - they're decoupled and flexible
• :green_square: Good, because easier to add additional context to obligations like time frames or other conditions driving feature behavior post-obligation
• :green_square: Good, because meets product use case requirements of "data-driven and derivable obligations satisfied by varied entity types under conditions within platform policy"
• :yellow_square: Neutral, because PEP and non-policy costs are close to the same
• :yellow_square: Neutral, because short-term policy implementation costs are close to the same, but maybe a little smaller
• :yellow_square: Neutral, because if we need hierarchy or allOf rules on obligations, we'd need a new enum construct, but it would be decoupled so they could evolve separately
• :red_square: Bad, because further complicates the policy relational db schema (hard to hold in your head at once)

Option 2: Obligations via flag within existing attributes as previously detailed

• :green_square: Good, because meets product use case requirements of "data-driven and derivable obligations satisfied by varied entity types under conditions within platform policy"
• :yellow_square: Neutral, because PEP and non-policy costs are close to the same
• :yellow_square: Neutral, because we may still be able to do things like derive obligations from entities, but it is going to be harder to understand as a developer if obligations are attributes
• :yellow_square: Neutral, because short-term policy implementation costs are close to the same, but maybe a little bigger
• :yellow_square: Neutral, because fewer tables is good, but coupling to attributes is limiting
• :red_square: Bad, because larger blast radius within policy for bugs and breaking changes given all the new IF/Else checks and behaviors
• :red_square: Bad, because less flexibility evolving obligations apart from attributes as our understanding of use cases matures
• :red_square: Bad, because implicit behavior around anyOf rule
• :red_square: Bad, because long-term costs are lower (maintainance, testing, onboarding)
• :red_square: Bad, because it's a greater burden on admins to understand (i.e. "here is what an attribute means, unless it's an obligation, and then it really means something different" and vice versa)

[ttschampel]

Option 1 makes sense to me especially if initial build cost is estimated to be the same between the two.

Is the obligation FQN naming convention what would be used to decipher the difference between data attribute and obligation if placed on a TDF data policy for purposes of access enforcement?

I think product should weigh in on whether a parent->child and obligation rule type would be necessary so this could be tackled in the initial API implementation and reduce API churn. It does seem like a handy way to do organization of "like" obligations and it seems like some obligations could be hierarchical

[jakedoublev]

@ttschampel Agreed, the underlying assumptions would need to be revised if parent/child on obligations and rule variations are expected to be utilized within policy and enforced appropriately.