kernel 6.6.4 gives kernel panic

[peabee]

kernel 6.6.3 was OK kernel 6.6.4 32-bit and 64-bit builds give kernel panic same system, same kernel booted with overlayfs, via punionfs=overlay boot switch, does not panic

[sfjro]

PB:

kernel 6.6.3 was OK kernel 6.6.4 32-bit build gives kernel panic

Thanks for the report. Currently I am busy and have no time for aufs. But when I get time, I will check linux-v6.6.4 ia32.

Unfortunately your panic log photo didn't give good information. '?' lines in the stack trace is unreliable. How is your configuration? The related ones I guess are

• CONFIG_FRAME_POINTER
• CONFIG_HAVE_RELIABLE_STACKTRACE (I guess this is yes)
• CONFIG_STACKTRACE

J. R. Okajima

[peabee]

CONFIG_FRAME_POINTER=y CONFIG_HAVE_RELIABLE_STACKTRACE is not set CONFIG_STACKTRACE=y

[sfjro]

"J. R. Okajima":

PB:

kernel 6.6.3 was OK kernel 6.6.4 32-bit build gives kernel panic

Thanks for the report. Currently I am busy and have no time for aufs. But when I get time, I will check linux-v6.6.4 ia32.

Glancing around v6.6.4 kernel source , I cannot see anything important to aufs. The changes are mainly for device drivers.

Could you try your 6.6.4 32-bit build WITHOUT aufs?

J. R. Okajima

[peabee]

Hi JRO

I did boot with overlayfs instead of aufs (everything else identical) and the boot worked with no panic.....

Both 32-bit & 64-bit 6.6.4 kernels panic when aufs is active.

A comment from Porteus user Beny says: Post#1942 by [beny] » 04 Dec 2023, 18:11 hi blaze the 6.6.4 kernel also with the kernel builder fail to add the devices to union, operation not permitted, but the build task is ok just for the record.

[roadie2]

JRO, The error I get is that directories can't be created in /mnt. The error is "makedir ........operation not permitted" It almost seems to be a permissions issue. I built a kernel using the kernel diff patch and 6.6 source files. I don't get a kernel panic, but I'm using a different initrd than peebee.

As mentioned, there are no errors with patching or compiling the kernel. As you said, most of the kernel diff file concerns drivers and I don't see what may be causing it, though I'm far from skilled in this.

[sfjro]

roadie2:

The error I get is that directories can't be created in /mnt. The error is "makedir ........operation not permitted" It almost seems to be a permissions issue. I built a kernel using the kernel diff patch and 6.6 source files. I don't get a kernel panic, but I'm using a different initrd than peebee.

Thanks for the info. Assuming your /mnt is aufs, did you set LSM (Linus Security Module)? If so, I'd ask you to try disabling or unsetting LSM.

J. R. Okajima

[roadie2]

JRO,

Apologies, the error I posted wasn't accurate, the error is : mkdir: can't create directory '/union/' : Operation not permitted I was not able to completely disable LSM in the kernel config, so resorted to having only apparmor. I've also tried variations of lsm= on the cmdline. It made no difference and repeated attempts with other options have not helped. Running ls -a shows that directory union is not present, yet running mkdir union gives an error that it already exists.

[fulalas]

I can confirm this issue. It seems the system fails to create /union for AUFS.

[sfjro]

fulalas:

I can confirm this issue. It seems the system fails to create /union for AUFS.

Thanx for the info. But I am going slightly confusing. Currently we have two issues.

Issue #31 from fulalas, Version 6.6: Kernel panic - not syncing: Can't create rootfs Issue #32 from peabee, kernel 6.6.4 gives kernel panic

They MAYBE rooted on the same problem, or at least related, or nothing related at all. I'm not sure.

I'll try one by one. At first, here is a debug patch for you fulalas. The base version is linux-v6.6. Please apply, build, boot, and report the kernel log. If 'pr_err' doesn't show the message, then try replacing by pr_crit, pr_alert, or pr_emerg. It depends on your log level setting. Note that this is a FIRST debug print patch to find the cause and never fix the problem. Such debug-print patch MAY continue a few times.

Thanx in advance.

J. R. Okajima

a.patch diff --git a/fs/fs_context.c b/fs/fs_context.c index 98589aae5208..e1c7a4da542e 100644 --- a/fs/fs_context.c +++ b/fs/fs_context.c @@ -186,6 +186,7 @@ int vfs_parse_fs_string(struct fs_context fc, const char key, }

ret = vfs_parse_fs_param(fc, &param);

• pr_err("vfs_parse_fs_param() %d\n", ret); kfree(param.string); return ret; } @@ -316,6 +317,7 @@ static struct fs_context alloc_fs_context(struct file_system_type fs_type, init_fs_context = legacy_init_fs_context;

  ret = init_fs_context(fc);

• pr_err("init_fs_context() %d\n", ret); if (ret < 0) goto err_fc; fc->need_free = true; @@ -716,6 +718,7 @@ int parse_monolithic_mount_data(struct fs_context fc, void data) if (!monolithic_mount_data) monolithic_mount_data = generic_parse_monolithic;

• pr_err("monolithic_mount_data -> %pf\n", monolithic_mount_data); return monolithic_mount_data(fc, data); }

diff --git a/fs/namespace.c b/fs/namespace.c index 1f68adec5b74..f106c7974177 100644 --- a/fs/namespace.c +++ b/fs/namespace.c @@ -1138,18 +1138,23 @@ struct vfsmount vfs_kern_mount(struct file_system_type type, return ERR_PTR(-EINVAL);

fc = fs_context_for_mount(type, flags);

• if (IS_ERR(fc))
• if (IS_ERR(fc)) {
• pr_err("fs_context_for_mount() %lu\n", PTR_ERR(fc)); return ERR_CAST(fc);
• } if (name) ret = vfs_parse_fs_string(fc, "source", name, strlen(name)); if (!ret) ret = parse_monolithic_mount_data(fc, data);
• else
• pr_err("vfs_parse_fs_string() %d\n", ret); if (!ret) mnt = fc_mount(fc);
• else
• else {
• pr_err("parse_monolithic_mount_data() %d\n", ret); mnt = ERR_PTR(ret);

• }

  put_fs_context(fc); return mnt; @@ -4691,8 +4696,10 @@ static void __init init_mount_tree(void) struct path root;

  mnt = vfs_kern_mount(&rootfs_fs_type, 0, "rootfs", NULL);

• if (IS_ERR(mnt))
• if (IS_ERR(mnt)) {
• pr_err("vfs_kern_mount() %lu\n", PTR_ERR(mnt)); panic("Can't create rootfs");

• }

  ns = alloc_mnt_ns(&init_user_ns, false); if (IS_ERR(ns)) diff --git a/init/do_mounts.c b/init/do_mounts.c index 5dfd30b13f48..64dec6d01115 100644 --- a/init/do_mounts.c +++ b/init/do_mounts.c @@ -496,9 +496,11 @@ void __init prepare_namespace(void) static bool is_tmpfs; static int rootfs_init_fs_context(struct fs_context *fc) {

• if (IS_ENABLED(CONFIG_TMPFS) && is_tmpfs)
• if (IS_ENABLED(CONFIG_TMPFS) && is_tmpfs) { +pr_err("here\n"); return shmem_init_fs_context(fc);
• } +pr_err("there\n"); return ramfs_init_fs_context(fc); }

[roadie2]

JRO, It appears that the issue is caused by /fs/stat.c The kernel-v6.6.4 is now booting well for me. I rather crudely replaced the stat.c file with one from kernel-v6.6.3 and it compiled without errors and is now booting.

Linux porteus.example.net 6.6.4-porteus #1 SMP Thu Dec 7 13:31:46 PST 2023 x86_64 AMD A10-9620P RADEON R5, 10 COMPUTE CORES 4C+6G AuthenticAMD GNU/Linux

quiet BOOT_IMAGE=/boot/syslinux/vmlinuz-6.6.4 from=/dev/sda2/porteus-xfce4-5.01 noauto mitigations=off norootcopy initrd=/boot/syslinux/initrd-5.01.zst

The relevant section begins at line 42299 of the 6.6.4.diff:

Removal of the " | AT_GETATTR_NOSEC" and "if (WARN_ON_ONCE(query_flags & AT_GETATTR_NOSEC)) return -EPERM; allows the kernel to boot.

[sfjro]

roadie2:

It appears that the issue is caused by /fs/stat.c The kernel-v6.6.4 is now booting well for me. I rather crudely replaced the stat.c file with one from kernel-v6.6.3 and it compiled without errors and is now booting.

Waoh, You found the root cause!! Now I can confirm that the commit 8a924db2d7b5 2023-11-18 fs: Pass AT_GETATTR_NOSEC flag to getattr interface function in linux-v6.7-rc4 and in v6.6.4 (3fb0fa086419) made the change. Aufs has to follow that change. I will release aufs6.6.4 as soon as possible.

Thank you very much.

J. R. Okajima

[peabee]

Well done @roadie2 - confirmed 6.6.4 built with reverted fs/stat.c boots correctly. Thanks a lot. Will test aufs-6.6.4 when it appears - thanks @sfjro

[sfjro]

------- Blind-Carbon-Copy

From: "J. R. Okajima" @.> To: @. Subject: aufs6 GIT release (v6.7 and v6.6.4) MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: @.> Date: Mon, 12 Feb 2024 04:59:58 +0900 Message-ID: @.>

Hello all,

Now I'm getting back to aufs slowly. Here are aufs6.7 and aufs6.6.4. Hopefully I'll start aufs6.x-rcN for linux-v6.8-rcN series in near future.

J. R. Okajima

---

• • aufs6-linux.git#aufs6.1..aufs6.6 nothing
• • aufs6-linux.git#aufs6.6.4
• • aufs6-linux.git#aufs6.7
• • aufs6-linux.git#aufs6.x-rcN v6.7-rc1, __mnt_drop_write() --> mnt_put_writeaccess() v6.7-rc1, accessor for inode i[am]time v6.7-rc3, aufs_getattr() supports AT_GETATTR_NOSEC flag
• • aufs6-standalone.git ditto
• • aufs-util.git nothing

------- End of Blind-Carbon-Copy

[peabee]

Many thanks @sfjro - greatly appreciated ;-))