search

cryfs / cryfs

Cryptographic filesystem for the cloud
https://www.cryfs.org
GNU Lesser General Public License v3.0
2.02k stars 155 forks source link

Force delete broken files #60

Open gador opened 8 years ago

gador commented 8 years ago

My computer crashed while accessing some files on the cryfs filesystem. Now I have the problem to be unable to delete some files.

When I access the directory no files are displayed in the filesystem browser. When accessing via CLI I get:

$ ls -la
ls: Zugriff auf '000002.pdy' nicht möglich: Eingabe-/Ausgabefehler
ls: Zugriff auf '000019.pdy' nicht möglich: Eingabe-/Ausgabefehler
ls: Zugriff auf '001266.pdy' nicht möglich: Eingabe-/Ausgabefehler
ls: Zugriff auf '000011.pdy' nicht möglich: Eingabe-/Ausgabefehler
ls: Zugriff auf '001269.pdy' nicht möglich: Eingabe-/Ausgabefehler
ls: Zugriff auf '000037.pdy' nicht möglich: Eingabe-/Ausgabefehler
ls: Zugriff auf '000004.pdy' nicht möglich: Eingabe-/Ausgabefehler
ls: Zugriff auf '001246.pdy' nicht möglich: Eingabe-/Ausgabefehler
insgesamt 0
drwxrwxr-x 1 florian florian 4096 Mai  5 08:57 .
drwxrwxr-x 1 florian florian 4096 Mai  5 08:57 ..
-????????? ? ?       ?          ?            ? 000002.pdy
-????????? ? ?       ?          ?            ? 000004.pdy
-????????? ? ?       ?          ?            ? 000011.pdy
-????????? ? ?       ?          ?            ? 000019.pdy
-????????? ? ?       ?          ?            ? 000037.pdy
-????????? ? ?       ?          ?            ? 001246.pdy
-????????? ? ?       ?          ?            ? 001266.pdy
-????????? ? ?       ?          ?            ? 001269.pdy

Zugriff auf '000002.pdy' nicht möglich: Eingabe-/Ausgabefehler: Translated meaning No Access to 000002.pdy. Input/output Error

While running cryfs in Foreground I get:


[2016-05-31 12:40:41.689] [cryfs] [error] Assertion [blob != boost::none] failed in /tmp/sourcedir-ced5b4dd/cryfs/src/cryfs/filesystem/parallelaccessfsblobstore/ParallelAccessFsBlobStore.h:56: Blob not found

cpputils::backtrace[abi:cxx11](): (cryfs+0x2b) [0x5c5efb]
cpputils::_assert::format(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, char const*, int): (cryfs+0x3e) [0x4f66ae]
cpputils::_assert::assert_fail_release(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, char const*, int): (cryfs+0x32) [0x4ff342]
std::_Function_handler<long (cpputils::FixedSizeData<16ul> const&), cryfs::parallelaccessfsblobstore::ParallelAccessFsBlobStore::_getLstatSize()::{lambda(cpputils::FixedSizeData<16ul> const&)#1}>::_M_invoke(std::_Any_data const&, cpputils::FixedSizeData<16ul> const&): (cryfs+0xd4) [0x56c224]
cryfs::fsblobstore::DirBlob::statChild(cpputils::FixedSizeData<16ul> const&, stat*) const: (cryfs+0x23) [0x567593]
cryfs::CryNode::stat(stat*) const: (cryfs+0x43) [0x568ff3]
fspp::FilesystemImpl::lstat(boost::filesystem::path const&, stat*): (cryfs+0x40) [0x590240]
fspp::fuse::Fuse::getattr(boost::filesystem::path const&, stat*): (cryfs+0x24) [0x5921f4]
cryfs() [0x592369]
: (/lib/x86_64-linux-gnu/libfuse.so.2+0xa218) [0x7f514f842218]
: (/lib/x86_64-linux-gnu/libfuse.so.2+0xa462) [0x7f514f842462]
: (/lib/x86_64-linux-gnu/libfuse.so.2+0x15679) [0x7f514f84d679]
: (/lib/x86_64-linux-gnu/libfuse.so.2+0x11e38) [0x7f514f849e38]
: (/lib/x86_64-linux-gnu/libpthread.so.0+0x76fa) [0x7f514f61f6fa]
clone: (/lib/x86_64-linux-gnu/libc.so.6+0x6d) [0x7f514e006b5d]

[2016-05-31 12:40:41.689] [cryfs] [error] AssertFailed in Fuse::getattr: Assertion [blob != boost::none] failed in /tmp/sourcedir-ced5b4dd/cryfs/src/cryfs/filesystem/parallelaccessfsblobstore/ParallelAccessFsBlobStore.h:56: Blob not found

cpputils::backtrace[abi:cxx11](): (cryfs+0x2b) [0x5c5efb]
cpputils::_assert::format(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, char const*, int): (cryfs+0x3e) [0x4f66ae]
cpputils::_assert::assert_fail_release(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, char const*, int): (cryfs+0x32) [0x4ff342]
std::_Function_handler<long (cpputils::FixedSizeData<16ul> const&), cryfs::parallelaccessfsblobstore::ParallelAccessFsBlobStore::_getLstatSize()::{lambda(cpputils::FixedSizeData<16ul> const&)#1}>::_M_invoke(std::_Any_data const&, cpputils::FixedSizeData<16ul> const&): (cryfs+0xd4) [0x56c224]
cryfs::fsblobstore::DirBlob::statChild(cpputils::FixedSizeData<16ul> const&, stat*) const: (cryfs+0x23) [0x567593]
cryfs::CryNode::stat(stat*) const: (cryfs+0x43) [0x568ff3]
fspp::FilesystemImpl::lstat(boost::filesystem::path const&, stat*): (cryfs+0x40) [0x590240]
fspp::fuse::Fuse::getattr(boost::filesystem::path const&, stat*): (cryfs+0x24) [0x5921f4]
cryfs() [0x592369]
: (/lib/x86_64-linux-gnu/libfuse.so.2+0xa218) [0x7f514f842218]
: (/lib/x86_64-linux-gnu/libfuse.so.2+0xa462) [0x7f514f842462]
: (/lib/x86_64-linux-gnu/libfuse.so.2+0x15679) [0x7f514f84d679]
: (/lib/x86_64-linux-gnu/libfuse.so.2+0x11e38) [0x7f514f849e38]
: (/lib/x86_64-linux-gnu/libpthread.so.0+0x76fa) [0x7f514f61f6fa]
clone: (/lib/x86_64-linux-gnu/libc.so.6+0x6d) [0x7f514e006b5d]

Is there a way to force the removal of said files and directories?

Cheers,

Florian

smessmer commented 8 years ago

Hm that sounds like your computer crashed while cryfs was in the middle of a file system operation that was modifying the file system. With a traditional filesystem like ext, you could use fsck or similar tools to recover from such a situation. Unfortunately, there is no such tool for cryfs available yet. The easiest way out for you would be to copy your files over to a new cryfs filesystem. Sorry about that.

On May 31, 2016 3:45:41 AM PDT, Florian notifications@github.com wrote:

My computer crashed while accessing some files on the cryfs filesystem. Now I have the problem to be unable to delete some files.

When I access the directory no files are displayed in the filesystem browser. When accessing via CLI I get:

$ ls -la
ls: Zugriff auf '000002.pdy' nicht möglich: Eingabe-/Ausgabefehler
ls: Zugriff auf '000019.pdy' nicht möglich: Eingabe-/Ausgabefehler
ls: Zugriff auf '001266.pdy' nicht möglich: Eingabe-/Ausgabefehler
ls: Zugriff auf '000011.pdy' nicht möglich: Eingabe-/Ausgabefehler
ls: Zugriff auf '001269.pdy' nicht möglich: Eingabe-/Ausgabefehler
ls: Zugriff auf '000037.pdy' nicht möglich: Eingabe-/Ausgabefehler
ls: Zugriff auf '000004.pdy' nicht möglich: Eingabe-/Ausgabefehler
ls: Zugriff auf '001246.pdy' nicht möglich: Eingabe-/Ausgabefehler
insgesamt 0
drwxrwxr-x 1 florian florian 4096 Mai  5 08:57 .
drwxrwxr-x 1 florian florian 4096 Mai  5 08:57 ..
-????????? ? ?       ?          ?            ? 000002.pdy
-????????? ? ?       ?          ?            ? 000004.pdy
-????????? ? ?       ?          ?            ? 000011.pdy
-????????? ? ?       ?          ?            ? 000019.pdy
-????????? ? ?       ?          ?            ? 000037.pdy
-????????? ? ?       ?          ?            ? 001246.pdy
-????????? ? ?       ?          ?            ? 001266.pdy
-????????? ? ?       ?          ?            ? 001269.pdy

Zugriff auf '000002.pdy' nicht möglich: Eingabe-/Ausgabefehler: Translated meaning No Access to 000002.pdy. Input/output Error

While running cryfs in Foreground I get:


[2016-05-31 12:40:41.689] [cryfs] [error] Assertion [blob !=
boost::none] failed in
/tmp/sourcedir-ced5b4dd/cryfs/src/cryfs/filesystem/parallelaccessfsblobstore/ParallelAccessFsBlobStore.h:56:
Blob not found

cpputils::backtrace[abi:cxx11](): (cryfs+0x2b) [0x5c5efb]
cpputils::_assert::format(char const*, std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&, char const*,
int): (cryfs+0x3e) [0x4f66ae]
cpputils::_assert::assert_fail_release(char const*,
std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> > const&, char const*, int): (cryfs+0x32)
[0x4ff342]
std::_Function_handler<long (cpputils::FixedSizeData<16ul> const&),
cryfs::parallelaccessfsblobstore::ParallelAccessFsBlobStore::_getLstatSize()::{lambda(cpputils::FixedSizeData<16ul>
const&)#1}>::_M_invoke(std::_Any_data const&,
cpputils::FixedSizeData<16ul> const&): (cryfs+0xd4) [0x56c224]
cryfs::fsblobstore::DirBlob::statChild(cpputils::FixedSizeData<16ul>
const&, stat*) const: (cryfs+0x23) [0x567593]
cryfs::CryNode::stat(stat*) const: (cryfs+0x43) [0x568ff3]
fspp::FilesystemImpl::lstat(boost::filesystem::path const&, stat*):
(cryfs+0x40) [0x590240]
fspp::fuse::Fuse::getattr(boost::filesystem::path const&, stat*):
(cryfs+0x24) [0x5921f4]
cryfs() [0x592369]
: (/lib/x86_64-linux-gnu/libfuse.so.2+0xa218) [0x7f514f842218]
: (/lib/x86_64-linux-gnu/libfuse.so.2+0xa462) [0x7f514f842462]
: (/lib/x86_64-linux-gnu/libfuse.so.2+0x15679) [0x7f514f84d679]
: (/lib/x86_64-linux-gnu/libfuse.so.2+0x11e38) [0x7f514f849e38]
: (/lib/x86_64-linux-gnu/libpthread.so.0+0x76fa) [0x7f514f61f6fa]
clone: (/lib/x86_64-linux-gnu/libc.so.6+0x6d) [0x7f514e006b5d]

[2016-05-31 12:40:41.689] [cryfs] [error] AssertFailed in
Fuse::getattr: Assertion [blob != boost::none] failed in
/tmp/sourcedir-ced5b4dd/cryfs/src/cryfs/filesystem/parallelaccessfsblobstore/ParallelAccessFsBlobStore.h:56:
Blob not found

cpputils::backtrace[abi:cxx11](): (cryfs+0x2b) [0x5c5efb]
cpputils::_assert::format(char const*, std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&, char const*,
int): (cryfs+0x3e) [0x4f66ae]
cpputils::_assert::assert_fail_release(char const*,
std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> > const&, char const*, int): (cryfs+0x32)
[0x4ff342]
std::_Function_handler<long (cpputils::FixedSizeData<16ul> const&),
cryfs::parallelaccessfsblobstore::ParallelAccessFsBlobStore::_getLstatSize()::{lambda(cpputils::FixedSizeData<16ul>
const&)#1}>::_M_invoke(std::_Any_data const&,
cpputils::FixedSizeData<16ul> const&): (cryfs+0xd4) [0x56c224]
cryfs::fsblobstore::DirBlob::statChild(cpputils::FixedSizeData<16ul>
const&, stat*) const: (cryfs+0x23) [0x567593]
cryfs::CryNode::stat(stat*) const: (cryfs+0x43) [0x568ff3]
fspp::FilesystemImpl::lstat(boost::filesystem::path const&, stat*):
(cryfs+0x40) [0x590240]
fspp::fuse::Fuse::getattr(boost::filesystem::path const&, stat*):
(cryfs+0x24) [0x5921f4]
cryfs() [0x592369]
: (/lib/x86_64-linux-gnu/libfuse.so.2+0xa218) [0x7f514f842218]
: (/lib/x86_64-linux-gnu/libfuse.so.2+0xa462) [0x7f514f842462]
: (/lib/x86_64-linux-gnu/libfuse.so.2+0x15679) [0x7f514f84d679]
: (/lib/x86_64-linux-gnu/libfuse.so.2+0x11e38) [0x7f514f849e38]
: (/lib/x86_64-linux-gnu/libpthread.so.0+0x76fa) [0x7f514f61f6fa]
clone: (/lib/x86_64-linux-gnu/libc.so.6+0x6d) [0x7f514e006b5d]

Is there a way to force the removal of said files and directories?

Cheers,

Florian

You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/cryfs/cryfs/issues/60

smessmer commented 8 years ago

Update: We're working on a way to auto-resolve synchronization conflicts, which would also auto-resolve such problems. It might take a bit before that feature is finished though.

jrwesolo commented 7 years ago

@smessmer Has there been any updates on this functionality?

smessmer commented 7 years ago

I'm still working on it, this is unfortunately not as easy as I thought initially.

jrwesolo commented 7 years ago

Understandable. Thanks for the update!

shalnoff commented 7 years ago

is there any risk to lost previously encrypted (already written down) files on accident power-down?

smessmer commented 7 years ago

If you're writing to the directory structure (e.g. adding new files) and your computer loses power, the directory structure could get corrupted. It's in theory possible to recover from that by rebuilding the directory structure, but there aren't any tools written for that yet.

shalnoff commented 6 years ago

@smessmer, thank you for the answer. Just to clarify... am I right understand that by loosing power I'm in risk to lost the whole encrypted directory, not just the single written file? and not just the subdirectory inside he encrypted directory?

smessmer commented 6 years ago

If the power goes out while you've been writing to the directory structure, you're at risk of losing the directory you've been making changes to, and all subdirectories below it. If the power goes out while you've been writing to a file, you're only at risk of losing that file. In the current version, writes happen more often than they should, because CryFS follows a strictatime policy, i.e. often writes to directories even if you're only reading stuff. This will change in the next version.

antymat commented 6 years ago

Is there any update on this? The issue is - it is not only when there is a power-down, it is also when there is some problem with the underlying fs - like with rclone mounted fs, sometimes the synchronisation fails and a file or directory is not stored. Then cryfs goes belly-up: I do have cryfses with directories I cannot remove other than by removing the whole cryfs. This is unfortunate - if I try and store 5TiB of data and then have to do it yet again, the whole idea loses its lustre.

zidz commented 4 years ago

This happened for me a couple of days ago. I'd overload cryfs in some way to find that the files was broken after a remount of the filesystem.

mareknovotny commented 4 years ago

what is the procedure to recover files on the one of 3 synchronized computers? I have only one of them crashed with error

CryFS Version 0.10.2

Password: 
Deriving encryption key (this can take some time)...done
[2020-07-18 19:20:24.424] [Log] [error] Crashed: Deserialization failed - missing nullbyte for string termination
mareknovotny commented 4 years ago

no good advice here?

mareknovotny commented 4 years ago

so answering my own question. Removed cryfs directory ~/.local/share/cryfs and then mount the volume again.