Can't properly mount NTFS folders into union using ntfs3

[fulalas]

When using ext partitions or squashfs files I can mount to union as read-only (rr) so that files can be edited in the union just fine (of course all changes are lost after reboot).

However, with NTFS partitions using ntfs3 driver (kernel 6.9.0 native), which allows POSIX compatibility, this doesn't seem to apply. In this case, a given mounted NTFS folder becomes read-only and I can no longer edit anything inside it in union. If I change the mount option to rw it becomes writable but then it gets whiteouts and all the mess I don't want.

That's the command I'm using: mount -no remount,add=1:"$NTFS_FOLDER"=rr aufs /union

I wonder why there's this difference between ext/squashfs and NTFS. Any workaround?

Thanks!

[sfjro]

fulalas:

However, with NTFS partitions using ntfs3 driver (kernel 6.9.0 native), which allows POSIX compatibility, this doesn't seem to apply. In this case, a given mounted NTFS folder becomes read-only and I can no longer edit anything inside it in union. If I change the mount option to rw it becomes writable but then it gets whiteouts and all the mess I don't want.

I'm afraid you might be misunderstanding the branch permision 'rr'. 'rr' stands for 'Real Read-only' and it is for the natively read-only filesystems such as iso9660, cramfs, romfs, or squashfs. It never affects the behaviour of the branch filesystem if you set 'rr' to your branch fs. If the branch fs is mounted as 'rw' (out of aufs mount), then the branch fs is still 'rw' after adding it into aufs mount and you can freely modify the files on it directly. For NTFS, 'rr' is equivalent to 'ro' since NTFS is not natively read-only.

If you do understand 'rr' correctly, then I'd ask you explain "I can no longer edit anything inside it in union" you wrote. Do you mean copy-up doesn't work?

J. R. Okajima

[fulalas]

@sfjro, thanks for your reply.

So you're saying that once mounted the result in AUFS filesystem is always going to be rw regardless?

What I'm facing here is odd because if I mount during boot the mounted folder is ro, but if I mount it after boot it's rw. Not sure what's happening tbh.

[sfjro]

fulalas:

So you're saying that once mounted the result in AUFS filesystem is always going to be rw regardless?

No. As long as you specify the branch attribute is RO at mounting aufs, it should keep RO.

What I'm facing here is odd because if I mount during boot the mounted folder is ro, but if I mount it after boot it's rw. Not sure what's happening tbh.

Still I don't understand yout situation. How is your /sys/fs/aufs/si_XXXX/brN? Does it show "=rw"? Do you mean

• you spcified the branch attribute is RO
• /sys/fs/aufs/si_XXXX/brN shows "=ro"
• but aufs writes to that branch, that is bad
• the branch fs is NTFS Right?

J. R. Okajima

[fulalas]

No. As long as you specify the branch attribute is RO at mounting aufs, it should keep RO.

My distro is mounting some squashfs files using this command:

mount -no remount,add=1:/memory/images/"$NAME"=rr aufs /union

The result in AUFS filesystem (union) is that I can edit/remove files normally -- of course, the squashfs files won't change.

How is your /sys/fs/aufs/si_XXXX/brN? Does it show "=rw"?

If I run this command cat /sys/fs/aufs/*/br* I get this:

/mnt/live/memory/changes=rw
/mnt/live/memory/images/firefox-esr-latest-115.10.0esr-x86_64-1.xzm=rr
/mnt/live/memory/images/003-xfce-4.18-current-20240505.xzm=rr
/mnt/live/memory/images/002-xtra-current-20240505.xzm=rr
/mnt/live/memory/images/002-gui-current-20240505.xzm=rr
/mnt/live/memory/images/001-core-current-20240505-ntfs3.xzm=rr
/mnt/live/memory/images/000-kernel-6.9.1.xzm=rr

As you can see, all squashfs (.xzm files) are set to read-only, however I can freely edit/delete them in AUFS. This has been like this since forever, and I never thought it would a problem. The actual problem I'm facing is that NTFS partitions don't behave the same way, as their mounted result in AUFS are read-only, which is not what I want.

[sfjro]

fulalas:

As you can see, all squashfs (.xzm files) are set to read-only, however I can freely edit/delete them in AUFS. This has been like this since forever, and I never thought it would a problem. The actual problem I'm facing is that NTFS partitions don't behave the same way, as their mounted result in AUFS are read-only, which is not what I want.

Hmm, it might be better to take a specific simple case to understand your situation.

• you have a writable branch as the first/topmost branch
• you had run "mount -o remount,add=1:"$NTFS_FOLDER"=ro /union" which is the second branch
• your ntfs contains "/dirA/fileB"
• both of dirA and fileB have writable permissions
• you tried editing the fileB in aufs (and you are expecting the internal copy-up behaviour)
• but it was refected by EACCES:Permission denied, EROFS:Read-only file system, or something. Am I right?

"Permissions" here includes all the ordinary permission bits, acl/xattrs, and LSM settings. What is the failed command you tried? Can you strace and find which systemcall returned the error?

J. R. Okajima

[fulalas]

I'm attaching 2 strace logs:

1. when I can execute mkdir build inside ~/ folder because I'm not mounting it from a NTFS partition: normal.txt

2. when this same call results in an error because I'm mounting ~/ from a NTFS folder (during boot time): ntfs.txt

[sfjro]

fulalas:

I'm attaching 2 strace logs:

Ok, the error happened at mkdir("build", 0777) = -1 EOPNOTSUPP (Operation not supported)

Let's try finding out the kernel internals.

• apply this patch to your source tree (which should be v6.9)
• rebuild aufs with CONFIG_AUFS_DEBUG is enabled, and append -DDEBUG to your CPPFLAGS
• reboot
• mount aufs including ntfs as its branch-fs
• make sure your topmost branch-fs is writable
• write "8" to /proc/sys/kernel/printk
• write "1" to /sys/fs/aufs/debug
• run "mkdir build"
• and check your kernel log

J. R. Okajima

diff --git a/fs/aufs/i_op_add.c b/fs/aufs/i_op_add.c index 3430838e0bc4..948ca4e2ee09 100644 --- a/fs/aufs/i_op_add.c +++ b/fs/aufs/i_op_add.c @@ -871,6 +871,9 @@ int aufs_mkdir(struct mnt_idmap idmap, struct inode dir, };

IMustLock(dir);

+AuDbgInode(dir); +AuDbgDentry(dentry); +AuDbg("idmap %p, mode 0%o\n", idmap, mode);

err = -ENOMEM;
a = kmalloc(sizeof(*a), GFP_NOFS);

@@ -878,9 +881,11 @@ int aufs_mkdir(struct mnt_idmap idmap, struct inode dir, goto out;

err = aufs_read_lock(dentry, AuLock_DW | AuLock_GEN);

+AuTraceErr(err); if (unlikely(err)) goto out_free; err = au_d_may_add(dentry); +AuTraceErr(err); if (unlikely(err)) goto out_unlock;

@@ -888,6 +893,7 @@ int aufs_mkdir(struct mnt_idmap idmap, struct inode dir, di_write_lock_parent(parent); wh_dentry = lock_hdir_lkup_wh(dentry, &a->dt, /src_dentry/NULL, &a->pin, &wr_dir_args); +AuTraceErrPtr(wh_dentry); err = PTR_ERR(wh_dentry); if (IS_ERR(wh_dentry)) goto out_parent; @@ -897,6 +903,7 @@ int aufs_mkdir(struct mnt_idmap idmap, struct inode dir, h_path.dentry = au_h_dptr(dentry, bindex); h_path.mnt = au_sbr_mnt(sb, bindex); err = vfsub_mkdir(au_pinned_h_dir(&a->pin), &h_path, mode); +AuTraceErr(err); if (unlikely(err)) goto out_unpin;

@@ -908,6 +915,7 @@ int aufs_mkdir(struct mnt_idmap idmap, struct inode dir, inode_lock_nested(h_inode, AuLsc_I_CHILD); opq_dentry = au_diropq_create(dentry, bindex); inode_unlock(h_inode); +AuTraceErrPtr(opq_dentry); err = PTR_ERR(opq_dentry); if (IS_ERR(opq_dentry)) goto out_dir; @@ -916,6 +924,7 @@ int aufs_mkdir(struct mnt_idmap idmap, struct inode dir, }

err = epilog(dir, bindex, wh_dentry, dentry);

+AuTraceErr(err); if (!err) { inc_nlink(dir); goto out_unpin; / success / diff --git a/fs/aufs/vfsub.c b/fs/aufs/vfsub.c index 62368b3e2730..19b475e5a65b 100644 --- a/fs/aufs/vfsub.c +++ b/fs/aufs/vfsub.c @@ -463,9 +463,11 @@ int vfsub_mkdir(struct inode dir, struct path path, int mode) goto out; idmap = mnt_idmap(path->mnt);

+AuDbg("dir->i_op->mkdir %pf\n", dir->i_op->mkdir); lockdep_off(); err = vfs_mkdir(idmap, dir, path->dentry, mode); lockdep_on(); +AuTraceErr(err); if (!err) { struct path tmp = *path; int did;

[fulalas]

I failed to build kernel with -DDEBUG, but at least CONFIG_AUFS_DEBUG is enabled. This is what I have after mkdir build:

aufs-debug.txt

[sfjro]

fulalas:

I failed to build kernel with -DDEBUG, but at least CONFIG_AUFS_DEBUG is enabled. This is what I have after mkdir build:

ok, good. Even without -DDEBUG, the log is good expectedly. But I made a mistake in the debug patch. Plz correct like this and run again.

fs/aufs/vfsub.c:vfsub_mkdir

466: AuDbgInode(dir); 467: AuDbg("dir->i_op->mkdir %ps\n", dir->i_op->mkdir); 468: lockdep_off(); 469: err = vfs_mkdir(idmap, dir, path->dentry, mode); 470: lockdep_on(); 471: AuTraceErr(err);

• AuDbgInode(dir) is added
• replace "%pf" by "%ps"

J. R. Okajima

[fulalas]

Sorry, it's not clear. Is this correct?

int vfsub_mkdir(struct inode *dir, struct path *path, int mode)
{
    int err;
    struct dentry *d;
    struct mnt_idmap *idmap;

    AuDbgInode(dir);
    AuDbg("dir->i_op->mkdir %ps\n", dir->i_op->mkdir);
    lockdep_off();
    err = vfs_mkdir(idmap, dir, path->dentry, mode);
    lockdep_on();
    AuTraceErr(err);

    d = path->dentry;
    path->dentry = d->d_parent;
    err = security_path_mkdir(path, d, mode);
    path->dentry = d;
    if (unlikely(err))
        goto out;
    idmap = mnt_idmap(path->mnt);

    AuDbg("dir->i_op->mkdir %ps\n", dir->i_op->mkdir);
    lockdep_off();
    err = vfs_mkdir(idmap, dir, path->dentry, mode);
    lockdep_on();
    AuTraceErr(err);
    if (!err) {
        struct path tmp = *path;
        int did;

        vfsub_update_h_iattr(&tmp, &did);
        if (did) {
            tmp.dentry = path->dentry->d_parent;
            vfsub_update_h_iattr(&tmp, /*did*/NULL);
        }
        /*ignore*/
    }

out:
    return err;
}

[sfjro]

fulalas:

Sorry, it's not clear. Is this correct?

No. Plz use this.

int vfsub_mkdir(struct inode dir, struct path path, int mode) { int err; struct dentry d; struct mnt_idmap idmap;

IMustLock(dir);

d = path->dentry;
path->dentry = d->d_parent;
err = security_path_mkdir(path, d, mode);
path->dentry = d;
if (unlikely(err))
    goto out;
idmap = mnt_idmap(path->mnt);

AuDbgInode(dir); AuDbg("dir->i_op->mkdir %ps\n", dir->i_op->mkdir); lockdep_off(); err = vfs_mkdir(idmap, dir, path->dentry, mode); lockdep_on(); AuTraceErr(err); if (!err) { struct path tmp = *path; int did;

    vfsub_update_h_iattr(&tmp, &did);
    if (did) {
        tmp.dentry = path->dentry->d_parent;
        vfsub_update_h_iattr(&tmp, /*did*/NULL);
    }
    /*ignore*/
}

out: return err; }

[fulalas]

Here we go :)

debug-new.txt

[sfjro]

fulalas:

Here we go :)

Xattr/Acl seems the trigger. Here is my currect guess.

• mkdir /home/you/build
• aufs tries copying-up /home/you from ntfs to topmost tmpfs
• the internal mkdir /home (or /home/you) on topmost tmpfs succeeded
• then aufs tries setting all attributes of just created dir, following the values on ntfs
• but tmpfs returns EOPNOTSUPP:Operation not supported, because
  • tmpfs doesn't support xattr/acl or
  • that attributes is unknown to tmpfs or
  • other reason

If this guess is right, then there are some approaches to fix. Choose one you like. You don't have to take all of them.

• make tmpfs support xattr/acl and accept any xattr/acl
• disable xattr/acl on ntfs and stop copy-up them
• make aufs to ignore the copyup error on that xattr/acl, using aufs' ICEX mount option.

(from aufs manual)

.B icexsec | icexsys | icextr | icexusr | icexoth | icex Ignore the error on copying-up/down XATTR. When an internal copy-up/down happens, aufs tries copying all XATTRs. Here an error can happen because of the XATTR support on the dst branch may different from the src branch. If you know how the branch supports or unsupports XATTR, you can specify these attributes. icexsec' means to ignore an error on copying\-up/down XATTR categorized as "security" (for LSM and capability). Andicexsys,' icextr,' and icexusr,' are for "system" (for posix ACL), "trusted" and "user" categories individually. icexoth' is for any other category. To be convenient,icex` sets them all. See also linux/Documentation/filesystems/aufs/design/06xattr.txt.

To know which xattr/acl caused the error, this debug patch may help.

diff --git a/fs/aufs/xattr.c b/fs/aufs/xattr.c index 6c83908fb380..26452f39404b 100644 --- a/fs/aufs/xattr.c +++ b/fs/aufs/xattr.c @@ -61,7 +61,10 @@ static int au_do_cpup_xattr(struct path h_dst, struct path h_src, struct mnt_idmap h_dst_idmap, h_src_idmap; struct posix_acl *acl;

+AuDbgDentry(h_src->dentry); +AuDbgDentry(h_dst->dentry); is_acl = !!is_posix_acl_xattr(name); +AuDbg("name %s, is_acl %d\n", name, is_acl); h_src_idmap = mnt_idmap(h_src->mnt); h_src_dentry = h_src->dentry; if (is_acl) { @@ -113,6 +116,7 @@ static int au_do_cpup_xattr(struct path h_dst, struct path h_src, }

out: +AuTraceErr(err); return err; }

J. R. Okajima

[fulalas]

To know which xattr/acl caused the error, this debug patch may help.

Using your suggested patch in combination with the other one (from previous debug log) resulted in a frozen boot. Here's the file, maybe I did something wrong:

xattr.txt

Gonna take a look at the documentation. Thanks a lot!

[sfjro]

fulalas:

Using your suggested patch in combination with the other one (from previous debug log) resulted in a frozen boot. Here's the file, maybe I did something wrong:

Then I'd suggest you to

• remove all patch I sent
• try getfattr, getfacl, getcap commands to your ntfs dir or file directly, to see what xattr they have

J. R. Okajima

[fulalas]

@sfjro, using 'icex' in the mount command seems to fix the issue.

I guess ntfs3 is not 100% POSIX compatible, but tbh I don't have time/energy to investigate why and report upstream. Let's hope ntfs3 gets better in the future. There are many issues still, which make it not suitable for production usage, in my opinion.

Thanks a loooot for your help and patience :)