search

jetwhiz / encfs4win

Windows port of EncFS
https://encfs.win
Other
401 stars 41 forks source link

Unreadable files when "Enable per-file initialization vectors" setting is on (reverse encryption) #47

Closed imperative closed 8 years ago

imperative commented 8 years ago

Environment

When I try to mount a filesystem into a virtual drive, it works as expected. The operation takes a couple of seconds and produces a readable reverse-enctrypted filesystem in the target drive. When I try instead to mount it into a folder on an already existing disk (have tried both the same and different disks than the one where the source folder is located), the process completes almost instantly, but the result does not work basically: the mountpoint folder now shows a proper-looking encrypted list of files, but trying to read any of the (reversely encrypted) files produces "file not found" error.

This seems like a very basic functionality that should of course be working, right? Am I doing something wrong?

Steps to reproduce problem

  1. intialize a reverse filesystem (producing the xml config file) encfs6.xml.txt
  2. Mount it (in --reverse mode of course), using non-privileged shell. mount --reverse /directory /mountpoint.

    Relevant logs

Attaching the logs, but the most relevant lines are probably 2016-07-12 00:12:20,686 VER [BlockNameIO.cpp:221] padding, _bx, finalSize = 202, 16, -170 2016-07-12 00:12:20,688 ERROR [default] withFileNode: error caught in getattr: invalid padding size Those lines come up every time an attempt to read a file is made. encfslog.txt

jetwhiz commented 8 years ago

@imperative -- The last time I saw an issue like this it was related to Block vs Block32 filename encoding. Based on your logs, it looks like you're using Block encoding (which is not compatible with Windows, since it is case-sensitive). Can you try to do a test with Block32 filename encoding and see if the problem still exists?

imperative commented 8 years ago

I have tried it now with the Block32 mode. The result is different: while previously trying to read file produced a 'read error' in an application that was trying to read it, now the file seems to open correctly, but opens up as empty. No contents is shown. Tried with different files, etc. Now the log instead produces following errors on each file read: 

    2016-07-18 22:23:08,103 ERROR [default] Assert failed: ino != 0
    2016-07-18 22:23:08,104 ERROR [default] withFileNode: error caught in read: ino != 0

I have already tried to remove all the previous version of the lib and dokan, restarting numerous times, and generally fiddling around with some parameters, but it is still not working.

Is this working for you? For example if you take the xml config file provided and try to mount it on your machine, it works?

jetwhiz commented 8 years ago

@imperative -- I'm not receiving that particular error on my end, but this might be related to an issue where the 8.3 alias is being used as the filename when a program requests to open it. In this case, a file saved as "MyFile.txt" will be opened as "MYFILE.TXT" (particularly if you're mounting into an existing volume instead of a new one, such as "Z:"). This issue does not exist when you mount to a new mount point (like "Z:"), correct?

Also, could you provide the full debug output for your new test (using Block32)?

imperative commented 8 years ago

Full debug log (including browsing into the mounted directory and trying to open one file). ecnfslogBlock32.txt The underlying filesystem for both the source files and for the new mountpoint is NTFS btw. What else can I try? Do you know where Assert failed: ino != 0 is from within the codebase? Maybe it can give me some clues too?

This issue does not exist when you mount to a new mount point (like "Z:")

I am not sure what exactly you mean by "new" and "old", but the issue exists when I try to mount it to a folder ("C:\some\folder"), no matter if that folder already exists or not (tried both). The issue does not exist if I try to mount into a "drive", like "Z:", or "S:", etc. I have only tried mounting into a non-existent drive letter, assuming that otherwise it would error out.

UPDATE: Actually, scratch that, after testing again now - in Block32 mode, even if I mount into a new drive (Z:), there is the same problem! Maybe this depends on it having already been mounted during this session, because during my earlier tests it wasn't like this. (Or maybe it's because of the Block32 mode.) The error in the log in this case is the same.

jetwhiz commented 8 years ago

The error is coming from this part of the code: https://github.com/jetwhiz/encfs4win/blob/master/encfs/CipherFileIO.cpp#L277

Can you try changing this line in encfs.cpp: https://github.com/jetwhiz/encfs4win/blob/master/encfs/encfs.cpp#L264

from: 

if (filler(buf, name.c_str(), NULL, 0)) break;

to: 

if (filler(buf, name.c_str(), &st, 0)) break;

You might have to build against the latest commit in Dokany for this to work (Dokany 1.0.0-RC3 does not have this fixed), otherwise readdir might have some issues (which is why we originally changed &st to NULL).

jetwhiz commented 8 years ago

Hi @imperative -- I have reverted the above change in our latest release (1.10.1-RC8) since it is no longer necessary with the newest version of Dokany. Can you confirm if this fixes the issue?

imperative commented 8 years ago

Unforunately, after uninstalling the encfs, and Dokany library, getting the latests releases from both (Dokany 1.0.0-RC4) it still does not work. Trying to read the files does not produce read errors, but the files are empty.

Relevant section from the log:

2016-08-03 17:52:45,156 VER [RawFileIO.cpp:115] open call, requestWrite = 0
2016-08-03 17:52:45,158 VER [RawFileIO.cpp:136] open file with flags 0, result = 4
2016-08-03 17:52:45,159 VER [encfs.cpp:653] encfs_open for [REDACTED], flags 0
2016-08-03 17:52:45,163 VER [CipherFileIO.cpp:465] handling reverse unique IV read: offset=0, dataLen=128
2016-08-03 17:52:45,165 ERROR [default] Assert failed: ino != 0
2016-08-03 17:52:45,166 ERROR [default] withFileNode: error caught in read: ino != 0
2016-08-03 17:52:45,168 VER [RawFileIO.cpp:115] open call, requestWrite = 0
2016-08-03 17:52:45,169 VER [RawFileIO.cpp:121] using existing file descriptor
2016-08-03 17:52:45,171 VER [encfs.cpp:636] NOTIFY: FIX Read only file switched to read only mode!
2016-08-03 17:52:45,173 VER [RawFileIO.cpp:115] open call, requestWrite = 0
2016-08-03 17:52:45,174 VER [RawFileIO.cpp:136] open file with flags 0, result = 4
2016-08-03 17:52:45,175 VER [encfs.cpp:653] encfs_open for [REDACTED], flags 0
2016-08-03 17:52:45,179 VER [CipherFileIO.cpp:465] handling reverse unique IV read: offset=0, dataLen=65536
2016-08-03 17:52:45,181 ERROR [default] Assert failed: ino != 0
2016-08-03 17:52:45,182 ERROR [default] withFileNode: error caught in read: ino != 0
2016-08-03 17:52:45,184 VER [RawFileIO.cpp:115] open call, requestWrite = 0
2016-08-03 17:52:45,186 VER [RawFileIO.cpp:121] using existing file descriptor
imperative commented 8 years ago

After making more tests with different settings I have managed to confirm that the problem lies solely in the Enable per-file initialization vectors? setting. If it is set to "no", everything works. Block32 mode, all other modes, mounting to a disk letter or to a ntfs folder on another disk. So the issue still exists and should be fixed, but obviously there is a workaround to disable that setting.

reverse encryption - chained IV and MAC disabled Enable per-file initialization vectors? This adds about 8 bytes per file to the storage requirements. It should not affect performance except possibly with applications which rely on block-aligned file io for performance. y/[n]: n

jetwhiz commented 8 years ago

Nice catch! It looks like the issue is caused by the inode number being improperly set on Windows. I have compiled a new version to fix this issue, which is available here:

https://dl.dropboxusercontent.com/u/50973674/encfs-installer.exe

Since the way inodes are computed has changed, it is possible that old encfs filesystems will not be compatible with this version (so you might have to re-encrypt). Note that NTFS is required to use per-file initialization vectors, since FAT32 does not have such a feature.

jetwhiz commented 8 years ago

My mistake -- you should not have to re-encrypt, since this issue affects reverse encryption only (with per-file IV). This has been fixed as of RC9.