google / recursive-version-control-system

Apache License 2.0
25 stars 7 forks source link

Encrypt large objects stored in the archive. #11

Closed ojarjur closed 1 year ago

ojarjur commented 1 year ago

This change modifies the storage of large objects in the archive so that they are encrypted before being written to the local file system.

The local file storage writes objects to two different subdirectories based on the object size; the objects subdirectory for objects up to 1MB in size, and the largeObjects subdirectory for any objects greater than 1MB.

This separation was intended to support using an eternal storage layer (such as an external disk or a network-attached-storage server) for storing the bulk of the archive so that it does not exhaust your local disk space.

However, if you are going to store these objects externally, then it's prudent to be more cautious about what is written to those external storage layers.

Accordingly, this change updates the file storage of large objects so that they are encrypted before being written to the largeObjects subdirectory.

The encryption is performed using the age encryption tool with the recipient being a locally generated identity stored in the archive (but outside of the largeObjects subdirectory).

Additionally, the filenames of these large objects are modified to include the suffix ".age" to distinguish them from unencrypted objects. That should make this change backwards compatible with existing archives, but at this stage in the project development we do not provide any guarantees of backwards compatibility.

This fixes https://github.com/google/recursive-version-control-system/issues/9