Prior art

[ianopolous]

Is this an appropriate place to mention prior art? That is, existing open source systems that already satisfy most, if not all, of the properties we are looking for in SDS?

I'm keen to avoid reinventing the wheel here.

[dmitrizagidulin]

@ianopolous definitely, please do! I think everyone in this group is very keen not to reinvent unnecessarily.

[bumblefudge]

I defer to the group on the order of operations, but it might be nice to write a paragraph each about the major prior art options, mentioning specifically which of the functional requirements each fails to meet, once that list is mostly finalized. This would be useful to readers of the spec with some, but not all, of the stated functional requirements that prefer older or easier-to-implement systems?

[ianopolous]

Okay great. I just wanted to point to a project (Peergos) that we've been working on for 7 years which has mostly the same requirements as SDS (Actually stronger privacy and security requirements in several cases). It's built on top of ipfs.

As a TLDR the properties we have include:

• global p2p private filesystem
• strong end to end encryption, and fine grained access control (read only and writable) (capability based, so server's are trustless)
• data model is hash linked data (IPLD) with updates signed by a keypair
• hide metadata from server including file names, mime-types, file sizes, directory topology, and also who has access to what (the server cannot see who has been granted access to what)
• directories are indistinguishable from small files to the server
• independent of DNS and TLS certificate authorities (though there is a web interface if you trust them)
• handle arbitrarily large files, including streaming, and O(1) seeking
• data can be trivially mirrored which provides live redundancy over the ipfs protocol
• efficient modification of large files
• resistant to quantum computer based attacks - unshared files are already fully post-quantum, shared files currently have a limited time window of vulnerability to a large quantum computer
• access control is done with a version of cryptree, improved to fit in the ipfs data model, which is a stunning data structure (https://github.com/Peergos/Peergos/raw/master/papers/wuala-cryptree.pdf)
• all data is in a merkle-champ (compressed hash-array mapped prefix-trie) which is a great data structure for content addressed mutable data, and plays very well with CRDTs as well.

You can read more in our book

[OR13]

@ianopolous I'd be in favor of a section of Prior art, in the main spec, where we can link to things this. Would you be willing to open a PR?

[ianopolous]

Sounds great, I'll do that asap.

[nikosft]

I think this paper is of interest to the group

S Jarecki, H Krawczyk, J Resch, "Updatable oblivious key management for storage systems," in ACM CCS 2019

You can download the paper here

The most interesting thing is that it allows key rotation without needing to re-encrypt the stored data. I am quoting from the paper:

[...] upon the rotation of a client’s key kc, server KmS computes a short update token ∆ as a function of the old and new keys kc, kc' and transmits ∆ to client C. Using ∆, C’s storage server StS can transform all ciphertexts that were encrypted with keys derived from kc into ciphertexts decryptable by the new kc' but not by the old kc.

The process is fast and it is based on OPRF which, AFAIK, is not rocket science.

[OR13]

@nikosft interesting... I'm familar with proxy re-encryption, and the thresholds required... and its always made me nervous ( I don't trust any system where a threshold of permissioned nodes are required, because NSLs exist, and so do bribes )... but I digress... can you try and ELI5 version of this paper on the issue for others?

[ianopolous]

@OR13 Just to clarify, is this the document I should be editing the prior art section into?

https://github.com/Peergos/secure-data-store/blob/master/index.html

[OR13]

@ianopolous yes.

[OR13]

@ianopolous can you open a PR to add a link, or we should close this issue.

[ianopolous]

Yep, sorry. I've been snowed under. I'll try and get to it soon.