search

google-code-export / s3ql

Automatically exported from code.google.com/p/s3ql
0 stars 0 forks source link

Support Access Control Lists (ACLs) #385

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Hi,
I see that a "cp -a" inside a s3ql fs doesn't copy the mode bits:
michele@server:/tmp/s3ql$ mount.s3ql --debug all --fg --compress zlib 
local:///tmp/s3ql/from/ /tmp/s3ql/to/
michele@server:/tmp/s3ql$ ls -l to/
drwx------ 1 michele michele 0  7 gen 09.26 lost+found
michele@server:/tmp/s3ql$ ls -l test
-rw-r--r-- 1 michele michele 0  8 gen 18.42 test
michele@server:/tmp/s3ql$ cp -av test to/
"test" -> "to/test"
michele@miclin:/tmp/s3ql$ ls -l to/
drwx------ 1 michele michele 0  7 gen 09.26 lost+found
-rw------- 1 michele michele 0  8 gen 18.42 test

As you can see, no mode bits has been copied, but chmod and chown works well.

Other strange thing it's that if I use "rsync -a" in place of "cp -a" it works 
correctly and mode bits are being copied!

Original issue reported on code.google.com by michele....@gmail.com on 8 Jan 2013 at 6:03

GoogleCodeExporter commented 9 years ago 
Interesting, thanks for the report. 

With cp:
2013-01-15 21:41:35.524 [15980] MainThread: [fs] lookup(1, 'test'): start
2013-01-15 21:41:35.524 [15980] MainThread: [fs] lookup(1, 'test'): start
2013-01-15 21:41:35.525 [15980] MainThread: [fs] create(id_p=1, test): started
2013-01-15 21:41:35.526 [15980] MainThread: [fs] setattr(4, [1358313540.812708, 
1358313530.2847977]): start
2013-01-15 21:41:35.526 [15980] MainThread: [fs] setxattr(4, 
'system.posix_acl_access', '\x02\x00\x00\x00\x01\x00\x06\x00\xff\xff\xff\xff\x
2013-01-15 21:41:35.527 [15980] MainThread: [fs] flush(4): start
2013-01-15 21:41:35.527 [15980] MainThread: [fs] release(4): start

And with rsync:

2013-01-15 21:41:37.325 [15980] MainThread: [fs] getattr(1): start
2013-01-15 21:41:37.325 [15980] MainThread: [fs] access(1): executed
2013-01-15 21:41:37.326 [15980] MainThread: [fs] lookup(1, 'rsynctest'): start
2013-01-15 21:41:37.326 [15980] MainThread: [fs] lookup(1, 'rsynctest'): start
2013-01-15 21:41:37.327 [15980] MainThread: [fs] lookup(1, 
'.rsynctest.D6TtVi'): start
2013-01-15 21:41:37.327 [15980] MainThread: [fs] create(id_p=1, 
.rsynctest.D6TtVi): started
2013-01-15 21:41:37.328 [15980] MainThread: [fs] setattr(5, [33152L]): start
2013-01-15 21:41:37.328 [15980] MainThread: [fs] flush(5): start
2013-01-15 21:41:37.328 [15980] MainThread: [fs] release(5): start
2013-01-15 21:41:37.328 [15980] MainThread: [fs] setattr(5, [1358314897.0, 
1358314225.0]): start
2013-01-15 21:41:37.329 [15980] MainThread: [fs] setattr(5, [33204L]): start
2013-01-15 21:41:37.329 [15980] MainThread: [fs] lookup(1, 'rsynctest'): start
2013-01-15 21:41:37.329 [15980] MainThread: [fs] rename(1, '.rsynctest.D6TtVi', 
1, 'rsynctest'): start
2013-01-15 21:41:37.330 [15980] MainThread: [fs] lookup(1, 
'.rsynctest.D6TtVi'): start
2013-01-15 21:41:37.330 [15980] MainThread: [fs] lookup(1, 'rsynctest'): start
2013-01-15 21:41:37.330 [15980] MainThread: [fs] forget([(5, 1)]): start

Original comment by Nikolaus@rath.org on 16 Jan 2013 at 5:44

GoogleCodeExporter commented 9 years ago 
While rsync correctly sets the mode of the copied file, cp instead sets the 
system.posix_acl_access extended attribute. If xattr support in the file system 
is disabled, cp falls back on setting the file mode bits and things appear to 
work fine.

The question is:
1) Is setting an ACL appropriate here? If not, it looks like a cp bug
2) Is there something wrong with the xattr handling in S3QL? Presumably, the 
value provided by cp should result in access rights identical to the source 
file.

Original comment by Nikolaus@rath.org on 17 Jan 2013 at 4:50

GoogleCodeExporter commented 9 years ago 
Ok, so the ACL is set correctly by cp. If you run "getfacl" on the copied file, 
it will have the right permissions.

It seems that for example ext4 automatically updates the permission bits when 
the ACL xattr is updated. Adding the same behaviour in S3QL would be 
non-trivial, as it would require parsing the ACL and extracting the subset that 
can be expressed in mode bits. In any case, this functionality should probably 
be implemented in FUSE, if not the kernel itself. I'll ask on the FUSE mailing 
list.

Original comment by Nikolaus@rath.org on 17 Jan 2013 at 5:05

GoogleCodeExporter commented 9 years ago 
This behaviour is a bug in FUSE. File permission bits and ACLs always have to 
be kept in sync by the file system. In-kernel file systems can do this easily 
using some convenience functions. FUSE, however, currently just forwards the 
raw setxattr()/setattr() calls, so it is up to the file system to keep ACL and 
file permissions in sync. Unfortunately this is very hard for the file system, 
because a file system sees ACL only as raw extended attribute values. 
Hardcoding any knowledge of this format seems like a very bad idea. 

This issue therefore needs to be addressed in FUSE itself. Ideally, FUSE will 
do all the required conversions and issue both setxattr() and setattr() calls 
as appropriate, so no additional logic is needed in the file system.

Until this is implemented, the best way to work around in S3QL is probably to 
refuse storing or reading of any ACL extended attributes. This will prompt 
applications like cp -a to fall back to the proper chmod/chown calls.

References: http://news.gmane.org/find-
root.php?message_id=%3c874nigs83s.fsf%40vostro.rath.org%3e and acl(5):

---snip---
CORRESPONDENCE BETWEEN ACL ENTRIES AND FILE PERMISSION BITS

     The permissions defined by ACLs are a superset of the permissions specified by the file per‐
     mission bits.

     There is a correspondence between the file owner, group, and other permissions and specific
     ACL entries: the owner permissions correspond to the permissions of the ACL_USER_OBJ entry. If
     the ACL has an ACL_MASK entry, the group permissions correspond to the permissions of the
     ACL_MASK entry.  Otherwise, if the ACL has no ACL_MASK entry, the group permissions correspond
     to the permissions of the ACL_GROUP_OBJ entry.  The other permissions correspond to the per‐
     missions of the ACL_OTHER_OBJ entry.

     The file owner, group, and other permissions always match the permissions of the corresponding
     ACL entry. Modification of the file permission bits results in the modification of the associ‐
     ated ACL entries, and modification of these ACL entries results in the modification of the
     file permission bits.
---snip---

Original comment by Nikolaus@rath.org on 22 Jan 2013 at 2:54

GoogleCodeExporter commented 9 years ago 
A workaround has been committed in revision 1c2cc70772b5. Now we need to wait 
for a FUSE bugfix, and then conditionally enable ACL support for recent-enough 
FUSE versions.

Original comment by Nikolaus@rath.org on 22 Jan 2013 at 3:19

GoogleCodeExporter commented 9 years ago

Original comment by Nikolaus@rath.org on 22 Jan 2013 at 3:19

GoogleCodeExporter commented 9 years ago 
My python2.7 (ubuntu precise) doesn't have ENOTSUP:
mount.s3ql --fg --compress zlib local:///test/from /test/to
Mounting filesystem...

now, ls -l /test/to

Encountered exception, trying to clean up...
Unmounting file system...
Uncaught top-level exception:
Traceback (most recent call last):
  File "/usr/bin/mount.s3ql", line 9, in <module>
    load_entry_point('s3ql==1.12', 'console_scripts', 'mount.s3ql')()
  File "/usr/lib/s3ql/s3ql/mount.py", line 139, in main
    llfuse.main(options.single)
  File "fuse_api.pxi", line 213, in llfuse.main (src/llfuse.c:18034)
  File "handlers.pxi", line 528, in llfuse.fuse_getxattr (src/llfuse.c:11337)
  File "handlers.pxi", line 529, in llfuse.fuse_getxattr (src/llfuse.c:11283)
  File "/usr/lib/s3ql/s3ql/fs.py", line 182, in getxattr
    raise FUSEError(errno.ENOTSUP)
AttributeError: 'module' object has no attribute 'ENOTSUP'

If I convert the return code from errno.ENOTSUP to errno.EOPNOTSUPP it doesn't 
raise errors and a "cp -a from to" test copy the attributes correctly.

Original comment by michele....@gmail.com on 31 Jan 2013 at 10:34

GoogleCodeExporter commented 9 years ago 
Thanks, will be fixed.

Original comment by Nikolaus@rath.org on 3 Feb 2013 at 4:58

GoogleCodeExporter commented 9 years ago 
Maybe we can do a bit better than return an error, cf. 
http://article.gmane.org/gmane.comp.file-systems.fuse.devel/12603

Original comment by Nikolaus@rath.org on 6 Feb 2013 at 3:52

GoogleCodeExporter commented 9 years ago 
Moved to 
https://bitbucket.org/nikratio/s3ql/issue/16/support-access-control-lists-acls

Original comment by Nikolaus@rath.org on 12 Jan 2014 at 3:54