and_cli and encfs on Mac OS El capitan

[nbittmann]

Hi there,

I've got a slight issue with the setup on my Mac and I'm unsure what's causing this:

I've mounted my amazon drive with encfs for encryption but after the first upload via cp or rsync the connection breaks and df -h reports 0bytes because the drive connection is lost

This is my current environment

#python3 -c 'import platform as p; print("%s\n%s" % (p.python_version(), p.platform()))'
3.5.2
Darwin-15.6.0-x86_64-i386-64bit

 #acd_cli version
acd_cli 0.3.2, api 0.9.2 

I run the following commands to mount the drives:

acd_cli -d -nl mount -i0 -fg /amazon/
 ENCFS6_CONFIG='/.encfs6.xml' encfs --extpass="/bin/echo ${encfs_pass}" /amazon/ /encfs/

And once the connection drops I get the following output I removed most of the first lines to make the output shorter since there is no actual useful info:

16-12-14 17:27:54.395 [DEBUG] [acdcli.acd_fuse] - -> write /NMDrEez1KD3F54e-5j6WqX4PiCE01j2LjIhkJQ4RgLHisOvuHkQtBNgNxMreK154SZ, (4096, 263192576, 7)
16-12-14 17:27:54.395 [DEBUG] [acdcli.acd_fuse] - <- write 4096
16-12-14 17:27:54.395 [DEBUG] [acdcli.acd_fuse] - -> write /NMDrEez1KD3F54e-5j6WqX4PiCE01j2LjIhkJQ4RgLHisOvuHkQtBNgNxMreK154SZ, (40, 263196672, 7)
16-12-14 17:27:54.395 [DEBUG] [acdcli.acd_fuse] - <- write 40
16-12-14 17:27:54.395 [DEBUG] [acdcli.acd_fuse] - -> write /NMDrEez1KD3F54e-5j6WqX4PiCE01j2LjIhkJQ4RgLHisOvuHkQtBNgNxMreK154SZ, (584, 263196672, 7)
16-12-14 17:27:54.395 [DEBUG] [acdcli.acd_fuse] - <- write 584
16-12-14 17:27:54.396 [DEBUG] [acdcli.acd_fuse] - -> flush /NMDrEez1KD3F54e-5j6WqX4PiCE01j2LjIhkJQ4RgLHisOvuHkQtBNgNxMreK154SZ, (7,)
16-12-14 17:27:57.286 [DEBUG] [acdcli.api.backoff_req] - Retry 0, waiting -61.349454s
16-12-14 17:27:57.286 [INFO] [acdcli.api.backoff_req] - PUT "https://content-eu.drive.amazonaws.com/cdproxy/nodes/{redacted}/content"
16-12-14 17:27:57.286 [DEBUG] [acdcli.api.backoff_req] - <MultipartEncoder: {('content', ('file.bin', <acdcli.api.content._TeeBufferedReader object at 0x11043b198>, 'application/octet-stream'))}>
16-12-14 17:27:57.288 [DEBUG] [requests.packages.urllib3.connectionpool] - Resetting dropped connection: content-eu.drive.amazonaws.com
send: b'PUT /cdproxy/nodes/{redacted}/content?suppress=deduplication HTTP/1.1\r\nHost: content-eu.drive.amazonaws.com\r\nAccept: */*\r\nUser-Agent: acdcli.api/0.9.2 python-requests/2.12.4\r\nAccept-Encoding: gzip, deflate\r\nConnection: keep-alive\r\nContent-Type: multipart/form-data; boundary={redacted}\r\nContent-Length: 263197443\r\nAuthorization: Bearer {redacted}\r\n\r\n'
send: <MultipartEncoder: {('content', ('file.bin', <acdcli.api.content._TeeBufferedReader object at 0x11043b198>, 'application/octet-stream'))}>
sendIng a read()able
16-12-14 17:28:18.700 [DEBUG] [acdcli.acd_fuse] - -> statfs / ()
16-12-14 17:28:18.700 [DEBUG] [acdcli.acd_fuse] - <- statfs {'f_blocks': 209715200, 'f_namemax': 256, 'f_frsize': 524288, 'f_bsize': 524288, 'f_bfree': 209706233, 'f_bavail': 209706233}
16-12-14 17:28:18.700 [DEBUG] [acdcli.acd_fuse] - -> statfs / ()
16-12-14 17:28:18.700 [DEBUG] [acdcli.acd_fuse] - <- statfs {'f_blocks': 209715200, 'f_namemax': 256, 'f_frsize': 524288, 'f_bsize': 524288, 'f_bfree': 209706233, 'f_bavail': 209706233}
16-12-14 17:28:27.285 [DEBUG] [acdcli.acd_fuse] - -> statfs / ()
16-12-14 17:28:27.285 [DEBUG] [acdcli.acd_fuse] - <- statfs {'f_blocks': 209715200, 'f_namemax': 256, 'f_frsize': 524288, 'f_bsize': 524288, 'f_bfree': 209706233, 'f_bavail': 209706233}
16-12-14 17:28:27.287 [DEBUG] [acdcli.acd_fuse] - -> statfs / ()
16-12-14 17:28:27.287 [DEBUG] [acdcli.acd_fuse] - <- statfs {'f_blocks': 209715200, 'f_namemax': 256, 'f_frsize': 524288, 'f_bsize': 524288, 'f_bfree': 209706233, 'f_bavail': 209706233}
16-12-14 17:28:29.320 [DEBUG] [acdcli.acd_fuse] - -> statfs / ()
16-12-14 17:28:29.320 [DEBUG] [acdcli.acd_fuse] - <- statfs {'f_blocks': 209715200, 'f_namemax': 256, 'f_frsize': 524288, 'f_bsize': 524288, 'f_bfree': 209706233, 'f_bavail': 209706233}
16-12-14 17:28:29.320 [DEBUG] [acdcli.acd_fuse] - -> statfs / ()
16-12-14 17:28:29.320 [DEBUG] [acdcli.acd_fuse] - <- statfs {'f_blocks': 209715200, 'f_namemax': 256, 'f_frsize': 524288, 'f_bsize': 524288, 'f_bfree': 209706233, 'f_bavail': 209706233}
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type header: Date header: Server header: x-amzn-RequestId header: Content-Length header: Connection 16-12-14 17:29:56.678 [DEBUG] [requests.packages.urllib3.connectionpool] - https://content-eu.drive.amazonaws.com:443 "PUT /cdproxy/nodes/{redacted}/content?suppress=deduplication HTTP/1.1" 200 666
16-12-14 17:29:56.678 [DEBUG] [acdcli.api.backoff_req] - x-amzn-RequestId:{redacted}
16-12-14 17:29:56.680 [INFO] [acdcli.cache.sync] - Inserted/updated 1 file(s).
16-12-14 17:29:56.682 [INFO] [acdcli.cache.sync] - Parented 1 node(s).
16-12-14 17:29:56.682 [INFO] [acdcli.cache.sync] - Applied properties to 1 node(s).
16-12-14 17:29:56.682 [DEBUG] [acdcli.acd_fuse] - <- flush None
16-12-14 17:29:56.715 [DEBUG] [acdcli.acd_fuse] - -> destroy / ()
16-12-14 17:29:56.715 [DEBUG] [acdcli.acd_fuse] - <- destroy None
mount_osxfuse: mount point /encfs is itself on a OSXFUSE volume
fuse: failed to mount file system: Invalid argument
fuse failed.  Common problems:
 - fuse kernel module not installed (modprobe fuse)
 - invalid options -- see usage message

EDIT:

Copying without encfs gives me the same issue so it seems to be something odd with acd_cli

My goal is not to have a local copy of the file and rather upload directly

Any suggestions appreciated

[pabru]

To the OP - Please think about using a different encryption tool. A recent audit found encfs to be insecure. More info on this is at https://defuse.ca/audits/encfs.htm .

[pink-mist]

That audit is 3 years old, so who knows how relevant it is these days, and it didn't have any actual exploit in it.

But let's take a look at the issues it raises:

1. Same key used for encryption and authentication. It lists both the security impact and exploitability as low. Authentication isn't something EncFS should really be in charge of - logging in to the system itself takes care of that. Irrelevant.

2. Stream cipher used to encrypt last file block. It lists exploitability as unknown and impact as high. Why? It gives a link to a reference that is no longer available, so I have no idea what that's about. It also complains that something it does with regards to this is "non-standard" and suggests to remove it completely and do something "more standard" instead. Being non-standard isn't a security issue in and of itself.

3. Generating block IV by XORing block number. It lists exploitability as low and impact as medium. This one seems fair enough, but still, exploitability is low. The solution it proposes was also quickly shot down, so there seems like there's not much to do.

4. File holes are not authenticated. Exploitability and impact are both listed as medium. I would consider them both low personally, but it depends on how you're using EncFS in the first place. Using it from an amazon cloud drive makes this virtually impossible to exploit.

5. 64-bit MACs. Exploitability listed as low, and impact as medium. This one I would say is valid and relevant, and exploitability should probably be higher than "low". I hope this has changed in the 3 years since this audit was done.

6. Editing configuration file disables MACs. Exploitability listed as high, impact as medium. As long as you don't store your encfs configfile on the Amazon Drive, I wouldn't consider this something to worry about.

In the end, there was one thing that worries me in using encfs on Amazon Drive, but which I expect has been fixed in the last 3 years, and one thing I didn't fully understand, because the link supposed to explain it wasn't working.

[pabru]

Hi,

Apologies for the wall of text, I hope this is readable enough. I just wanted to reply some of the points you have raised in your previous message.

TL;DR: The most concerning issue this audit raised seems to be 2. IMO, and I don't know if that's been fixed yet. Basically, 2. is all about the use of DIY crypto algorithms, which definitely should not be used - unless you are a crypto expert on par with authors of stuff like AES or Twofish (see long version for more). In any case, I believe that encfs should fix all of the issues pointed out in that audit and re-check their approach to cryptography throughout the codebase to ensure they are up to standard with modern crypto practices.

Long version:

1. This report is using the word "authentication" in its cryptographic sense, not the usual "access to the system" case. Authentication is crypto is basically responsible for proving message integrity and that nobody changed any of the data. What the author said there was that it's considered a bad idea to use the same key for both encryption and authentication/integrity checking. I'm only a little bit knowledgeable in crypto, but apparently there are reasons why you want to avoid it. For example, at this Security.SE topic, one answer says that this may make it easier to break the encryption - if the same key is used for both, and the authentication used is broken, you automatically know the encryption key too. I'm not saying this is the case with encfs, but there are reasons why you would not want to share keys that way.
2. The first reference is basically used as evidence that the stream cipher is used at all, so that part is OK. The concerning part is that encfs is basically using custom encryption for some of its data. One of the basic rules of crypto is that you Do Not Roll Your Own Crypto. Bruce Schneier (who designed Twofish and Blowfish, one of the strongest and most widespread ciphers worldwide) explains the reason behind this in a very good way (IMO):

Anyone, from the most clueless amateur to the best cryptographer, can create an algorithm that he himself can't break. It's not even hard. What is hard is creating an algorithm that no one else can break, even after years of analysis. And the only way to prove that is to subject the algorithm to years of analysis by the best cryptographers around.

Basically unless you are a crypto expert, you should be picking something pre-designed by crypto experts.

3. Here we should be probably looking at the "Security Impact" factor, which is listed as Medium. Basically he's saying that encfs re-uses IVs, which is a bad idea because in cryptography, they are specifically meant to not be re-used. Unfortunately I can't explain much further than that, I'm not knowledgeable enough in that area.
4. Agreed, I would also consider this as low importance personally. Basically all this one would allow you to do is to insert random strings of zeroes in the files at any point (AFAIK without changing data otherwise), which would probably only annoy you and/or mean you have to repair your files/restore from backup depending on the actual file formats and so on.
5. Agreed. If you forge the MAC (authentication), then you can basically modify encrypted files without the user noticing, which is not something to allow.
6. Agreed, you probably should keep the config file on your machine. If someone compromises it there, it's too late anyway.

Personally I'm using either FDE (although I'm a bit more nervous now that I've read the You Don't Want XTS article linked in the audit), or the default rclone encryption settings (for cloud backups), or the archive-based default AES encryption of 7zip's 7z format.

I was bitten in the back once by file-based encryption of my home directory, but if I ever needed to use something like that, I would probably test out eCryptFS as it seems to be a decent alternative to encfs.

[pink-mist]

Thanks, your explanation was very enlightening :)