Replication questions

[tahpot]

We had a robust conversation about CouchDB on our last call with some people in favour and some people against, but I didn't feel we came away a whole lot clearer.

As a follow up, I would like to pose the following questions:

• What other replication + sync implementations exist today that could be considered for inspiration?
• With what we know up to this point in the EDV spec, what are the major technical blockers against using a P/CouchDB implementation? @csuwildcat ?
• What are the pros / cons of creating a whole new replication implementation and spec for confidential storage vs taking an existing implementation like CouchDB and "standardizing" it?

[mwherman2000]

What other replication + sync implementations exist today that could be considered for inspiration?

Replication prior-art (not necessarily re-usable technology but certainly re-usable patterns)

Partitioned by Content Change Detection/Notification Model

1. CRUD Event based
   • Syntergy Replicator for SharePoint: Replication Pipeline: Outbound Processing, Package Transfer, Inbound Processing
   • Groove Workspace
2. Update Sequence Number (USN) based
   • Microsoft Active Directory
3. Crawler Events based
   • Don't know of any - technically not valid option

[Update] The reason why a CRUD Events model or a USN model work is because they create a time-sequenced list of content changes/transactions. For example, in the general case, if a file f lives in a folder F, on the target server you have to make sure that the creation of F occurs before f is placed into it. Similarly, if you use ACLs, the resource (e.g. a folder or a file) has to exist before the ACLs can be associated with it. Similarly, for content resources that are dependent on schema resources, the schema resource has to be placed in storage before it referenced by a subsequent content resource, etc. etc. Trying to address these situations with a plethora of "business rules" is next to impossible. Time-sequenced changes/transactions is the way to go.

Running queries against an existing corpus of resources doesn't work for the same reason ...except for very specific scenarios.

[mwherman2000]

We had a robust conversation about CouchDB

@tahpot Are you referring to the entire CouchDB vertically integrated database platform as a potential (or potential starting point for an) EDV implementation?
Reference: https://docs.couchdb.org/en/stable/intro/overview.html

...or are you more specifically referring to just the CouchDB Replication Protocol? Reference: https://docs.couchdb.org/en/stable/replication/protocol.html

[tahpot]

...or are you more specifically referring to just the CouchDB Replication Protocol? Reference: https://docs.couchdb.org/en/stable/replication/protocol.html

I am interested in Daniel's specific thoughts on CouchDB generally, but in the context of this conversation I'm more referring to the CouchDB Replication Protocol.

[agropper]

Regardless of how little of the CouchDB Replication Protocol we adopt for EDV, would it be possible for CouchDB to be compliant as an EDV?

[mwherman2000]

Regardless of how little of the CouchDB Replication Protocol we adopt for EDV, would it be possible for CouchDB to be compliant as an EDV?

@agropper You would have to do an assessment of CouchDB's capabilities level-by-level against the CS spec:

• Layer A - Secure Content Storage Containers
• Layer B - Secure Content Storage Service
• Layer C - Authorization Service

...or using the old (current spec) terminology (https://identity.foundation/confidential-storage/#ecosystem-overview)...