mxmlnkn / ratarmount

Access large archives as a filesystem efficiently, e.g., TAR, RAR, ZIP, GZ, BZ2, XZ, ZSTD archives
MIT License
709 stars 36 forks source link

Potential issue with python 3.7 #127

Closed michaelgtodd closed 7 months ago

michaelgtodd commented 8 months ago

My python is a little weak to ID exactly what's happening here, but I noticed that when mounting a write overlay with python 3.7.5 and ratarmount 0.14, and then trying to add a new file I was getting:

Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/fuse.py", line 495, in _wrapper
    return func(*args, **kwargs) or 0
  File "/usr/lib/python3.7/site-packages/fuse.py", line 511, in getattr
    return self.fgetattr(path, buf, None)
  File "/usr/lib/python3.7/site-packages/fuse.py", line 759, in fgetattr
    attrs = self.operations('getattr', self._decode_optional_path(path), fh)
  File "/usr/lib/python3.7/site-packages/fuse.py", line 800, in __call__
    return getattr(self, op)(*args)
  File "/usr/local/lib/python3.7/site-packages/ratarmount.py", line 670, in getattr
    return self._fileInfoToDict(self._getFileInfo(path))
  File "/usr/local/lib/python3.7/site-packages/ratarmount.py", line 632, in _getFileInfo
    raise fuse.FuseOSError(fuse.errno.ENOENT)
AttributeError: module 'fuse' has no attribute 'errno'

from the debug foreground log, and prints like

touch: cannot touch 'test': Bad address

from the terminal trying to add the file.

Experimentally I tried adding

import * from errno

to ratarmount.py, and then replacing all of the instances of say

fuse.errno.ENOENT

with for instance

ENOENT

Which did resolve the issue, but I imagine this isn't optimal.

mxmlnkn commented 8 months ago

Which did resolve the issue, but I imagine this isn't optimal.

It should be fine. It seems to be a code modification that shouldn't change the observable behavior. It might nevertheless be interesting:

As for the first issue, could you please paste the output of ratarmount --version?

michaelgtodd commented 8 months ago
ratarmount my_archive.tar ./my_archive --write-overlay ./write-overlay
Python 3.7.5
FUSE 2.9.9
libsqlite3 3.26.0

Compression Backends:

indexed_bzip2 1.4.1
rapidgzip 0.10.4
indexed_gzip 1.8.7
xz 0.4.0
indexed_zstd 1.1.3
rarfile 4.1

This is on Fedora 29 (super back dated for support for another project) so maybe I have a weird remix of other distros.

mxmlnkn commented 8 months ago

I can tell you that I have FUSE 3.10.5, maybe pre 3.0 versions didn't have fuse.errno. I might have to check.

But, this doesn't explain the underlying issue. That one sounds more like maybe missing write permissions in the overlay folder or something like that. ENOENT is raised when it doesn't find the requested file. I'm assuming, you did a touch my_archive/test after the mount process. Unfortunately, I cannot reproduce the error with your command line. I also tried removing write permissions from the overlay folder, but then I got an expected "Permission denied" error when trying to create a file in the mount point.

michaelgtodd commented 8 months ago

Ah, it may be crucial that this is creating a new file - so test doesn't exist when the touch command is executed, or when the code goes to find it. I'll admit I didn't follow where the code goes after that, but I assumed someone must traditionally catch the exception, note that a new file is being created, and pursue some other flow. Since the seg fault is from the '--debug 3' log, and the write fail is from another log I don't know which happens first. I can say that with the changes to reference errno exactly, I was able to not only create the file in the overlay, but then commit the overlay afterwards.

mxmlnkn commented 8 months ago

I assumed someone must traditionally catch the exception

The FuseOSError exceptions are caught be fusepy (the ratarmount code is inside fuse.FUSE, which then calls the ratarmount implementation and wraps all of them in try-catch blocks) and then should be converted into a file system error, e.g., "No such file or directory".

I can say that with the changes to reference errno exactly, I was able to not only create the file in the overlay, but then commit the overlay afterwards.

Thanks for the explanation. So everything works fine after this change. Then, I'll change the import fuse.errno to import errno, which should be fine, and then I this should be resolved. The thrown exception is normal error signaling behavior between ratarmount and fusepy.