A LevelUP compatible leaderless multi-master database with eventual consistency, using hyperbee + CRDT + HLC. Similarly CockroachDB achieves replication on top of RocksDB, but here it is a pure P2P **streaming** database, with zero central management. LevelDB compatibility allows to use Dynalite on top to achieve DynamoDB compatibility with sophisticated auto-updated secondary indexes, and fairly complex queries. Work on Dynalite is almost completed to remove HTTP server, to make this combination perfect for serverless.
I discussed permissions with @Mauveranger, @martinheidegger and @pgmemk . Specifically how to do:
permissions on individual objects in hyperbee and on individual files in hyperdrive
(obviously we pay the cost of replicating the file to all peers, even those that do not have permission). This can be mitigated by creating multiple hyperdrives (not convenient of course)
permissions for whole devices (e.g. device added / lost)
permissions on whole hypercore
Permissions on objects
Let's first consider read permissions
We came to the conclusion that we can't avoid replicating those object but can encrypt them and selectively give the keys only to certain peers (maybe in the future @mafintosh will figure out if protocol-level messages like can be used to limit access to some peers, but it may not be easy as protocol does not have notion of hypertrie / hyperbie / hyperdrive objects).
The question is how to remove permissions for something you shared already? We will follow the Google Docs model here. What has been shared could have been downloaded, copied and send to others without asking for additional permissions. There is nothing we can do about it. But we can:
stop sharing new updates on the object
remove this object from the store, so that hyperbee and hyperdrive never return it. This way for user it disappears.
can we clear() object from hypercore versions ?
So how do we share updates with some but not the others?
Apparently there is a simple algo used by encrypted email for years.
Summary
you encrypt data once, but then encrypt the encryption key many times, each for a target user, which represents the permission for them to read the object. Note that this is a granular per-object encryption key, not per whole feed. Overhead it minuscule, assuming you do not have millions of peers to distribute to. So it is not for Netflix use case, it is for Google Drive use case.
Details
Generate a random symmetric key S1
Encrypt the entire message with S1, creating the cyphertext C1
Encrypt S1 with each public key for the recipients, generating D1, D2, ... , Dn
Generate a secure hash of S1, creating H1
Generate the message as follows:
-- BEGIN CYPHERTEXT --
C1
-- END CIPHERTEXT --
-- BEGIN HASH --
H1
-- END HASH --
-- BEGIN KEYS --
D1
D2
D3
-- END KEYS--
Send the message to everybody. They will decrypt D1 to Dn to produce S1 candidates. Then they has each S1 candidate and compare them with H1. If one is equal, they got the S1.
With the symmetric key on hand, they can independently decrypt the message, without knowing each other keys.
Write permissions
This can be implemented with other peers not accepting writes from the unauthorized writer.
UI will also need some API support to indicate to user an ability to edit, and to allow editing.
Different types of permissions
I discussed permissions with @Mauveranger, @martinheidegger and @pgmemk . Specifically how to do:
Permissions on objects
Let's first consider read permissions
We came to the conclusion that we can't avoid replicating those object but can encrypt them and selectively give the keys only to certain peers (maybe in the future @mafintosh will figure out if protocol-level messages like can be used to limit access to some peers, but it may not be easy as protocol does not have notion of hypertrie / hyperbie / hyperdrive objects).
The question is how to remove permissions for something you shared already? We will follow the Google Docs model here. What has been shared could have been downloaded, copied and send to others without asking for additional permissions. There is nothing we can do about it. But we can:
So how do we share updates with some but not the others?
Apparently there is a simple algo used by encrypted email for years.
Summary
you encrypt data once, but then encrypt the encryption key many times, each for a target user, which represents the permission for them to read the object. Note that this is a granular per-object encryption key, not per whole feed. Overhead it minuscule, assuming you do not have millions of peers to distribute to. So it is not for Netflix use case, it is for Google Drive use case.
Details
Send the message to everybody. They will decrypt D1 to Dn to produce S1 candidates. Then they has each S1 candidate and compare them with H1. If one is equal, they got the S1.
With the symmetric key on hand, they can independently decrypt the message, without knowing each other keys.
Write permissions
This can be implemented with other peers not accepting writes from the unauthorized writer. UI will also need some API support to indicate to user an ability to edit, and to allow editing.
TBD.