search

digitalbazaar / edv-client

An Encrypted Data Vault Client
BSD 3-Clause "New" or "Revised" License
13 stars 9 forks source link

Consider sending stream sequence and number of chunks to server #78

Closed dlongley closed 3 years ago

dlongley commented 3 years ago

If we remove this line that deletes the stream sequence and number of chunks:

https://github.com/digitalbazaar/edv-client/blob/6bdadb3999140fed1fd9fa33ebc7c9c29e649d4d/EdvClient.js#L1092

We could transmit this information to the server, which is useful for replication and archival purposes. The server can already determine the sequence number and number of chunks if it pays attention to writes to chunk indexes out of band, so this is not technically extra knowledge, it is just more conveniently "in band". We should be careful not to send content hashes to the server if/when we implement those in the stream meta data -- unless those hashes are on the encrypted data.

cc: @dmitrizagidulin, @msporny

What are your thoughts @OR13? We may make this (breaking) change soon and include this information in a new major release of the client.

OR13 commented 3 years ago

@dlongley we and secure key have not implemented streams yet, so the more breaking changes you can make before that, the better :)

I am not sure of the implications, but it sounds like this is metadata that the server should have, and there is no reason for the client not to send it.

I am not sure about the comment regarding ciphertext digests, is the idea that the client would compute the digest and send the digest along with the chunks? isn't this redundant given the http digest header?

dlongley commented 3 years ago

@OR13,

Thanks for the feedback.

I am not sure about the comment regarding ciphertext digests, is the idea that the client would compute the digest and send the digest along with the chunks? isn't this redundant given the http digest header?

Yes, I think this would be more about digests on plaintext -- I just want to make sure that we're careful about what we send up as string meta data. We should be explicit in what gets sent rather than forwarding along whatever is in stream.

@dmitrizagidulin -- we should proceed with implementing this when we have time. We should replace deleting stream by instead setting it to contain the sequence and number of chunks (the two fields that are currently present, but we should explicitly list them out so if we add other fields in the future they don't get sent without a review).

OR13 commented 3 years ago

worth noting that digests of plaintext is a feature that @cswildcat is interested in for hubs.

in particular that the digest of the initial plaintext might be an acceptable identifier for the document.

dmitrizagidulin commented 3 years ago

Addressed by PR #80 and PR https://github.com/digitalbazaar/bedrock-edv-storage/pull/69. Closing.