Epic 552 - Decentralize data model

[VassilIordanov]

• [x] Make all IDs to be universal - #562
• [ ] Add a SHA1 hash and timestamp fields
• [ ] ID + hash should be unique
• [ ] design a way to designate which revisions have been approved, through an approval chain similar to reports; see #199 for example

[VassilIordanov]

Requirements

1. Design a universal method for handling Entities (Reports, People, Places, Organizations, Positions, etc)
2. An entity should have a life cycle beyond the data store of its creation. For example, it should be easily moved from one store to another without the need for complex manipulations
3. Two datastores, with or without a common ancestor should be able to be merged, provided with a conflict resolution strategy, without the need for complex manipulations
4. A datastore should support non destructive edits

Proposed Approach

Vocabulary

• entity - an abstract data structure identified by a uuid
• version - a version of an entity is an immutable object instance with the uuid of the entity, a timestamp, a state (draft/published) a SHA-1 hash of the object content, and entity subtype-specific attributes. It is identified by uuid + hash
  • published version - a version that has been approved and can be viewed by the public.
  • draft version - a version that has not yet been approved and is typically only visible to its author and the approving chain
• datastore - a collection of entity versions. It can be typically resolved to a consistent graph of the latest versions prior to a point in time - or the current time for the most up to date state.

Data structures

Example of a Person entity

{
    "http://anet.org/entity/uuid": "63ee47c7-b56f-4847-9f91-bd7416d5ee63",
    "http://anet.org/entity/timeStamp": " ",
    "http://anet.org/entity/hash": "64AA9268A1949A76343085A4D6494B5AD5EBE6BE",
    "http://anet.org/entity/state": "published",

    "http://anet.org/person/name": "SMITH, John",
    "http://anet.org/country": "Suburbia",
    "http://anet.org/telephone": "123 3454"
}

Example of a Position entity

{
    "http://anet.org/entity/uuid": "7f778bc2-1f2c-4bf2-95a9-160f1b3fd8ba",
    "http://anet.org/entity/timeStamp": " ",
    "http://anet.org/entity/hash": "64AA9268A1949A76343085A4D6494B5AD5EBE6BE",
    "http://anet.org/entity/state": "published",

    "http://anet.org/position/name": "Director",
    "http://anet.org/position/personRef": "63ee47c7-b56f-4847-9f91-bd7416d5ee63"
}

Possible datastructure technologies

• https://json-ld.org/

Semantic behavior

• a uuid is generated when a new entity is created
• in each datastore, uuid+hash is unique
• in each datastore, only one version can exist per published report
• old/stale draft versions could either be garbage collected and/or compressed
• cached graph of the published entities. An in-memory partial or full cache could be maintained on the client and/or server side (likely partial/lazy on client).
  • given a point in time, the cache could provide a traversable graph consisting of the most recent published versions prior to that point
  • given a uuid, the cache could provide the list of versions of that UUID
• Graphql behavior:
  • current behavior should remain, by default the latest published version should be returned
  • query params could be extended to also take a point in time
  • not sure yet how to gracefully make queries return multiple versions