search

sfjro / aufs-standalone

27 stars 14 forks source link

Aufs for Linux 5.8 ETA? #2

Closed fatalhalt closed 3 years ago

fatalhalt commented 3 years ago

Hello,

Will there be aufs patchset released for Linux 5.8 anytime soon? Thanks.

sfjro commented 3 years ago

Hello,

fatalhalt:

Will there be aufs patchset released for Linux 5.8 anytime soon? Thanks.

Unfortunately, the situation is unchanged from https://www.mail-archive.com/aufs-users@lists.sourceforge.net/msg05822.html

J. R. Okajima

peabee commented 3 years ago

I wish J.R.O @sfjro a speedy resolution of his problems and a return to his greatly appreciated work for the community. in the meantime.... This is very very unofficial and only to be used at your own risk..... but you can find an amended aufs5-base.patch at: https://u.pcloud.link/publink/show?code=XZI5x7XZH0G8B1h38uRzXngfAhAqChaW2XoV which does allow 5.8 to be built with aufs-5.x-rcN The resulting 5.8 built for Puppy Linux seems to work fine in that environment - but only that environment has been "tested"....

fatalhalt commented 3 years ago

@peabee thanks for the patch! It is working fine on my custom gentoo distro with 5.8.11 kernel . 🙂

peabee commented 3 years ago

5.9 is likely to be the next LTS kernel .... so probably more "important" than 5.8. Looking to the future, I decided it would be interesting to see if 5.9-rc6 with aufs would build. Further changes to the aufs patches were required - in both aufs5-base.patch and aufs5-mmap.patch Then the patched kernel failed to compile throwing an error in fs/aufs/hfsnotify.c Having made a change to hsnotify.c (described below) kernel 5.9-rc6 did build. A Puppy Linux system with the new kernel booted successfully (screenshot below) ..... large programs such as chromium and libreoffice could be installed and run from sfs (squashfs) files.... HOWEVER the filemanager PcManFM would not run (but didn't throw any errors either). (the other file filemanager RoxFiler did run OK)

I've probably got the fix for the hsnotify.c compile error completely wrong but due to lack of knowledge I do not know what the correct fix should be.

The compile error indicated that at line 205 of hsnotify.c there were [-Werror=incompatible-pointer-types]

205 | .handle_event = au_hfsn_handle_event,

and in particular the order of the components were slightly different so as "au_hfsn_handle_event" was declared at line 162, I changed the position of "struct inode *inode," to match ".handle_event" so that it read:

static int au_hfsn_handle_event(struct fsnotify_group group, u32 mask, const void data, int data_type, struct inode inode, const struct qstr file_name, u32 cookie, struct fsnotify_iter_info *iter_info)

Anybody (with more knowledge) able to suggest what the correct fix might be? Thanks!

Screenshot

peabee commented 3 years ago

5.9 has been released but is not yet identified as an LTS kernel.... https://www.kernel.org/

peabee commented 3 years ago

OK - have built 5.9 with the changes previously tried on 5.9-RC6 and this time they seem to have worked successfully..... pcmanfm now works as expected. I've uploaded the 3 aufs files that had to be changed: https://u.pcloud.link/publink/show?code=XZvhPFXZG6bEshssryQQO150kW2kPYwrBWQk ... Screenshot

peabee commented 3 years ago

5.10 assistance required........ https://www.zdnet.com/article/linux-5-10-finally-ditches-decades-old-tool-that-caused-security-bugs/ : "Torvalds drew attention to the removal of an addressing tool, called set_fs()" this causes problems in https://github.com/sfjro/aufs5-standalone/blob/aufs5.7/fs/aufs/xino.c e.g. "fs/aufs/xino.c:894:2: error: implicit declaration of function ‘set_fs’; did you mean ‘sget_fc’?"

Is there a simple replacement for set_fs and get_fs in xino.c ??

sfjro commented 3 years ago

Hi PB,

PB:

5.10 assistance required........=0D ::: Is there a simple replacement for set_fs and get_fs in xino.c ??=0D

I will return to aufs development as soon as possible and dive into this issue. For a while, will you try these options?

  1. According to the commit 3c57fa13f6bf 2020-10-04 asm-generic: make the set_fs implementation optional Enabling CONFIG_SET_FS looks hopeful. Did you already try it?

  2. fs/read_write.c also uses set_fs(). How does it handle this? Can we simply follow the same say? $ git diff v5.8 v5.10-rc4 read_write.c :::

    • old_fs = get_fs();
    • set_fs(KERNEL_DS);
    • p = (force const char user *)buf;
    • if (count > MAX_RW_COUNT)
    • count = MAX_RW_COUNT;
    • if (file->f_op->write)
    • ret = file->f_op->write(file, p, count, pos);
    • else if (file->f_op->write_iter)
    • ret = new_sync_write(file, p, count, pos);
    • else
    • ret = -EINVAL;
    • set_fs(old_fs);
    • init_sync_kiocb(&kiocb, file);
    • kiocb.ki_pos = pos ? *pos : 0;
    • iov_iter_kvec(&iter, WRITE, &iov, 1, iov.iov_len);
    • ret = file->f_op->write_iter(&kiocb, &iter); :::

J. R. Okajima

fredx181 commented 3 years ago

Hi J. R. Okajima

I will return to aufs development as soon as possible and dive into this

Great news! And many thanks for all your great work during the years.

techrockedge commented 3 years ago

I agree with fredx181, good news and also thank you for the excellent work.

peabee commented 3 years ago

Hello @sfjro Great to hear from you and for your advice - many thanks. I have tried putting CONFIG_SET_FS=y into the kernel config but this doesn't seem to be sufficient as the compile time errors still occur .... the key include referring to CONFIG_SET_FS seems to be linux-5.10.0/include/asm-generic/uaccess.h but I don't know if xino.c will be including this. Your Option 2 seems to require a major rewrite of xino.c (there are 8 occurrences of set_fs and 4 occurrences of get_fs) which I fear is beyond my capabilities! Thanks again peebee

peabee commented 3 years ago

Notes: As well as xino.c cpup.c vfsub.c also use set_fs & get_fs and need to be updated. cpup.c seems to be different to the other 2 and does not relate to the read_write.c model. seq_file.c in the kernel also uses init_sync_kiocb(&kiocb, file);

sfjro commented 3 years ago

"J. R. Okajima":

I will return to aufs development as soon as possible and dive into this issue. For a while, will you try these options?

Here is a patch for linux-v5.10-rc5. It passed the subset of my local tests. Why subset? Because my full test may take much longer time and I don't have enough time for that.

Please try

J. R. Okajima

peabee commented 3 years ago

Hi @sfjro Many thanks but....

I don't think you included the patch?????

sfjro commented 3 years ago

PB:

I don't thinkyou included the patch?????

I did attached, but I cannot see it on github.com either. Hmm, github doesn't accept xz compressed attachment, or no attachement at all? Now I try attaching bz2 compressed patch.

J. R. Okajima

peabee commented 3 years ago

@sfjro - still invisible - maybe try renaming with -false.gz ??

sfjro commented 3 years ago

Now a.patch.gz.

Then a.patch.gz again, but in different mail format.

J. R. Okajima

sfjro commented 3 years ago

PB:

@sfjro - still invisible - maybe try renaming with -false.gz ??

Hmm, something is wrong. Really wrong. Am I? Now a.patch.gz.

peabee commented 3 years ago

Patch sent to me by email - many thanks.

peabee commented 3 years ago

Splendid news with great thanks due to J.R.Okajima ( @sfjro ) for finding the time to produce the quite complicated changes needed to integrate aufs5 with kernel 5.10-rc5. I have been able to build a working aufs-enabled kernel 5.10-rc5 with the patches sent to me by email. I have uploaded to: https://sourceforge.net/projects/lxpup/files/Other/huge-kernels/kernel-5.10-aufs-changes/aufs-5.10-changed-files.tar.xz/download .... three amended aufs patches of the kernel sources - aufs5-base.patch aufs5-mmap.patch aufs5-standalone.patch - to substitute the patches downloaded from Github aufs5.x-rcN .... plus, based on the @sfjro supplied new patch, - a new k510.patch - which has to be applied to the kernel sources (including aufs) after aufs has been added to the stock kernel. As kernel 5.10 has been confirmed at the next Long Term Supported LTS kernel release, this is indeed good news. Cheers PeeBee

sfjro commented 3 years ago

PB:

Splendid news with great thanks due to J.R.Okajima ( @sfjro ) for finding the time to produce the quite complicated changes needed to integrate aufs5 with kernel 5.10-rc5. I have been able to build a working aufs-enabled kernel 5.10-rc5 with the patches sent to me by email.

Thank you for testing and publishing. When I fully come back to aufs development, I will start from aufs-5.8. So reaching aufs-5.10 may take some time. Please be patience.

J. R. Okajima

peabee commented 3 years ago

Kernel 5.10 has been released. aufs-enabled version builds apparently successfully with the above patches.

arekm commented 3 years ago

Loopback patch needs update, too. This hunk:

--- drivers/block/loop.c~       2020-12-15 14:31:38.000000000 +0100
+++ drivers/block/loop.c        2020-12-15 14:36:32.548939602 +0100
@@ -1189,6 +1189,7 @@ static int loop_configure(struct loop_de
        lo->use_dio = lo->lo_flags & LO_FLAGS_DIRECT_IO;
        lo->lo_device = bdev;
        lo->lo_backing_file = file;
+       lo->lo_backing_virt_file = virt_file;
        lo->old_gfp_mask = mapping_gfp_mask(mapping);
        mapping_set_gfp_mask(mapping, lo->old_gfp_mask & ~(__GFP_IO|__GFP_FS));
peabee commented 3 years ago

Hi @arekm Is the above patch for 5.8 5.9 or 5.10 ? What problems do you see without this patch? Thanks

arekm commented 3 years ago

Original loopback patch no longer applies to 5.10. And above hunk is a fixed version. With this 5.10 builds fine for me (+ patches from this issue).

peabee commented 3 years ago

Thanks - we don't apply the loopback patch so didn't detect the need for this further change - see next comment.

peabee commented 3 years ago

Original loopback patch no longer applies to 5.10. And above hunk is a fixed version. With this 5.10 builds fine for me (+ patches from this issue).

Hi @arekm Are you saying that the loopback.patch in aufs-5.x-rcN should be deleted and that the only content of loopback.patch for 5.10 should be the single patch that you have posted above?

There are many other files patched in the aufs-5.x-rcN loopback.patch other than drivers/block/loop.c .......

arekm commented 3 years ago

No. If you try to apply loopback patch then one hunk will fail. That one hunk needs to be replaced with above.

peabee commented 3 years ago

Hi @arekm Can you email me the complete amended loopback.patch that works with 5.10 please?

peabee commented 3 years ago

I have updated the 5.10 changes download with the loopback.patch sent to me by @arekm (with thanks). However I haven't been able to check the amended patch fully. I did note however that it still contains patches for a function 'static int loop_set_fd' which isn't present in the 5.10 drivers/block/loop.c however this doesn't seem to raise any errors.

sfjro commented 3 years ago

PB:

I have updated the 5.10 changes download with the loopback.patch sent to me by @arekm (with thanks). However I haven't been able to check the amended patch fully. I did note however that it still contains patches for a function 'static int loop_set_fd' which isn't present in the 5.10 drivers/block/loop.c however this doesn't seem to raise any errors.

It might be v5.8 instead of v5.10. By the commit in v5.8-rc1, 3448914e8cc55 2020-05-21 loop: Add LOOP_CONFIGURE ioctl aufs5-loopback.patch looks need to be updated. And I think Arkadiusz's patch is correct, but I haven't test it by myself.

FYI, I attach my local untested aufs5-loopback.patch. Arkadiusz, is this perfectly match with yours?

Oh, wait. Github looks discarding my mail attachment. I will append the patch as a text after my signature.

J. R. Okajima

SPDX-License-Identifier: GPL-2.0 aufs5.x-rcN loopback patch

diff --git a/drivers/block/loop.c b/drivers/block/loop.c index 8d77e68aa52b9..655f2252187a8 100644 --- a/drivers/block/loop.c +++ b/drivers/block/loop.c @@ -648,6 +648,15 @@ static inline void loop_update_dio(struct loop_device *lo) lo->use_dio); }

+static struct file loop_real_file(struct file file) +{

@@ -722,12 +732,19 @@ static int loop_change_fd(struct loop_device lo, struct block_device bdev, file = fget(arg); if (!file) goto out_err;

@@ -739,6 +756,7 @@ static int loop_change_fd(struct loop_device lo, struct block_device bdev, blk_mq_freeze_queue(lo->lo_queue); mapping_set_gfp_mask(old_file->f_mapping, lo->old_gfp_mask); lo->lo_backing_file = file;

@@ -1085,7 +1107,7 @@ static int loop_configure(struct loop_device lo, fmode_t mode, struct block_device bdev, const struct loop_config *config) {

@@ -1208,6 +1237,8 @@ static int loop_configure(struct loop_device *lo, fmode_t mode, bd_abort_claiming(bdev, claimed_bdev, loop_configure); out_putf: fput(file);

diff --git a/drivers/block/loop.h b/drivers/block/loop.h index af75a5ee40944..1d847cb194ff6 100644 --- a/drivers/block/loop.h +++ b/drivers/block/loop.h @@ -46,7 +46,7 @@ struct loop_device { int (ioctl)(struct loop_device , int cmd, unsigned long arg);

diff --git a/fs/aufs/f_op.c b/fs/aufs/f_op.c index 6fb4a4ed8cc7f..ba9a959f2db27 100644 --- a/fs/aufs/f_op.c +++ b/fs/aufs/f_op.c @@ -359,7 +359,7 @@ static ssize_t aufs_read_iter(struct kiocb kio, struct iov_iter iov_iter) if (IS_ERR(h_file)) goto out;

@@ -36,6 +38,8 @@ AuStubVoid(au_warn_loopback, struct super_block *h_sb)

AuStubInt0(au_loopback_init, void) AuStubVoid(au_loopback_fin, void) + +AuStub(struct file , aufs_real_loop, return NULL, struct file file)

endif / BLK_DEV_LOOP /

endif / KERNEL /

diff --git a/fs/aufs/super.c b/fs/aufs/super.c index 589dd01220201..801e0a7faec59 100644 --- a/fs/aufs/super.c +++ b/fs/aufs/super.c @@ -844,7 +844,10 @@ static const struct super_operations aufs_sop = { .statfs = aufs_statfs, .put_super = aufs_put_super, .sync_fs = aufs_sync_fs,

peabee commented 3 years ago

Congrats & well done @sfjro ...........

@fatalhalt this issue can be closed ............

Subject aufs5 GIT release (v5.{8,9,10}) From J. R. Okajima To aufs-users@lists.sourceforge.net Date Today 16:21 I'm back. Thanks for your patience and sorry for you inconvenient. I hope I can keep my development pace as I had.

o news

J. R. Okajima