feature suggestion: ecryptfs file format

[GoogleCodeExporter]

Instead of using OpenPGP or a proprietary format for file encryption, Duplicati 
may use eCryptFS. From it's web page: "eCryptfs stores cryptographic metadata 
in the header of each file written, so that encrypted files can be copied 
between hosts; the file will be decryptable with the proper key, and there is 
no need to keep track of any additional information aside from what is already 
in the encrypted file itself. Think of eCryptfs as a sort of ``gnupgfs.''" 
http://launchpad.net/ecryptfs

Compared to GnuPG eCryptFS has the advantage that the encrypted files can be 
mounted similar to any other cryptographic file system.

Original issue reported on code.google.com by j...@suhr.info on 3 Jul 2010 at 11:50

[GoogleCodeExporter]

I like it, but I see two problems:
1) It is linux only, and relies heavily on linux features (keyring etc).
2) It doesn't support compression.

The volumes created by Duplicati do not contain many easily readable files, as 
they are usually just diffs, so the benefit from being able to mount seems low 
IMO.

The recent previews now use the AESCrypt format, which is well documented and 
decryptable using either Duplicati or the AESCrypt tools.

I won't dedicate time for this right now, but I like the idea of supporting 
another cryptosystem, so I'll leave it here.

If you disagree with my point of view, let me know.

Original comment by kenneth....@gmail.com on 5 Jul 2010 at 8:01

• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

Regarding your first point: "1) It is linux only, and relies heavily on linux 
features (keyring etc)."
You are right that currently it relies heavily on Linux. But I don't see a 
technical reason for this and thus it may change in the future. For example 
nowadays exist a driver for the Linux-originated crypto file system 
dm-crypt/LUKS. Maybe the situation may change for ecryptfs in the future.
Regarding the keyring example I don't understand why this is Linux-specific. 
For my understanding this is out-of-scope of the filesystem specification but 
depends on the implementation.

Regarding your second aspect: "2) It doesn't support compression." This is 
right. A solution could be to apply any compression algorithm additionally. But 
a user who would mount the ecryptfs system directly would see "herfile.gz" for 
instance instead of "herfile". This is not a perfect solution but may be 
acceptable. You may also ask the ecryptfs developers what they think about a 
compression feature in ecryptfs.

Original comment by j...@suhr.info on 20 Jul 2010 at 3:06

[GoogleCodeExporter]

The current assumption in Duplicati is that "compression" handles a bunch of 
files, and "encyption" handles a single file. The reason for this is that it is 
how most encryption/compression tools work. In this respect eCryptFS is a bit 
different, and perhaps more like a light version of LUKS or TrueCrypt. This 
distinction does not allow eCryptFS to "fit in" as easily as another 
cryptosystem.

1) 
You may be right that the keyring integration is optional, and that it is 
portable. As I read the description though, it appears to depend on some kernel 
features, that I don't think can be invoked on another OS. Of course there is 
no wrong in having a module that only works under Linux, so I revoke my 
"problem 1" :D.

2)
It looks as if the output from the eCryptFS is a folder with encrypted files 
and folders. To reliably transfer this to another host, say WEBDAV, it needs to 
be wrapped in a single file, so a transfer is either complete or not, never 
half-complete. After encryption, the files contain random data, which is not 
compressible according to entropy rules. Encrypting each file separately also 
adds overhead in the form of header data to each file, making the result larger 
than what is possible with the "compress then encrypt" system.

If eCryptFS got support for compression, it would still mean that there was a 
bunch of files, which would have to be tar'ed together to provide a single 
transfer.

If there is a benefit to having eCryptFS data rather than zip.aes/zip.gpg, this 
is perhaps a minor issue.

3)
The content of a Duplicati volume is not simple files. The full backup does 
contain the complete files (as the name suggests), but in the current format, 
they may be split over several volumes, making it tedious to extract them 
manually. The incremental volumes only contain diff data, which again may be 
split. 

If you were to mount a volume, you may be able to see files from the full 
backups, but not much else that would be usefull.

Given this, I don't see the benefit from being able to mount the volumes. It 
will not enable you to quickly extract a given file, because Duplicati stores 
the changes from the original to the current, unlike rsync, which stores the 
current and then changes back to the original. Rsync can do this because it can 
actually run code on the target host, which is not possible with many of the 
supported backends in Duplicati.

If eCryptFS supported remote filesystems, eg. NFS, it would be very cool to 
make an rsync to a remote eCryptFS folder. This would enable you to quickly get 
the most recent version of the file, and with the help of rsync, also an older 
version.

Original comment by kenneth....@gmail.com on 20 Jul 2010 at 6:57

[GoogleCodeExporter]

Ok, I see your points and agree that currently eCryptFS does not have huge 
benefits which make it worth to deal with the described inadequacy. But maybe 
the situation will change in the future.

> If eCryptFS supported remote filesystems, eg. NFS, it would be very cool
> to make an rsync to a remote eCryptFS folder. This would enable you to
> quickly get the most recent version of the file, and with the help of
> rsync, also an older version.

Without any specific eCryptFS support you can already apply the following 
scenarios:

A) Having a remote ecryptfs directory, mount it on the remote server and share 
it via NFS for instance. In this case you have to trust the server because it 
will get the private key.

B) Put a ecryptfs encrypted folder on any remote storage e.g. NFS and mount it 
locally. Here you don't need to trust the server.

Not sure if I caught your point...

Original comment by j...@suhr.info on 21 Jul 2010 at 12:49