Open ernsteiswuerfel opened 1 year ago
Hmm, this is working for me when running on a Debian ppc64 porterbox:
(sid_ppc64-dchroot)tytso@perotto:~/e2fsprogs/e2fsprogs/build/tests$ ./test_script m_rootdir_acl
m_rootdir_acl: create fs image from dir using inline_data and acls: ok
374 tests succeeded 0 tests failed
(sid_ppc64-dchroot)tytso@perotto:~/e2fsprogs/e2fsprogs/build/tests$ uname -a
Linux perotto 6.4.0-1-powerpc64 #1 SMP Debian 6.4.4-1 (2023-07-23) ppc64 GNU/Linux
By the way, I don't know how long I will have access to a ppc64 Big Endian machine. From: https://github.com/golang/go/issues/34850:
IBM has been pushing very hard to deprecate ppc64 and move everyone over to ppc64le and we're not spending any additional time or effort on supporting ppc64. The "newest" OS for ppc64 we have which still has support is Ubuntu 16.04 or CentOS 7. Anything beyond that is only ppc64le. Debian dropped ppc64 support in Stretch (9) and Ubuntu dropped it in 18.04. Fedora also no longer has ppc64 AFAIK.
So far, the OSUOSL machine is still around, and Debian is still running autobuilders for ppc64 and s390x, but s390x is the only remaining "officially" supported big endian machine left which is officially supported by Debian. Right now ppc64 (big endian) is only supported unofficially, which I suspect means the "bus factor" for ppc64 is very high....
Wanted to verify my results on ppc32 and ran the tests again with git-master on my ppc32 partition on the same machine, kernel is also exactly the same.
To my surprise _m_rootdiracl passed!
Filesystem volume name: <none>
Last mounted on: <not available>
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg inline_data sparse_super huge_file dir_nlink extra_isize metadata_csum
Default mount options: (none)
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 1024
Block count: 16384
Reserved block count: 819
Overhead clusters: 1799
Free blocks: 14533
Free inodes: 1003
First block: 1
Block size: 1024
Fragment size: 1024
Group descriptor size: 64
Reserved GDT blocks: 127
Blocks per group: 8192
Fragments per group: 8192
Inodes per group: 512
Inode blocks per group: 256
Flex block group size: 16
Mount count: 0
Check interval: 15552000 (6 months)
Reserved blocks uid: 0
Reserved blocks gid: 0
First inode: 11
Inode size: 512
Required extra isize: 32
Desired extra isize: 32
Journal inode: 8
Default directory hash: half_md4
Journal backup: inode blocks
Checksum type: crc32c
Journal features: (none)
Total journal size: 1024k
Total journal blocks: 1024
Max transaction length: 1024
Fast commit length: 0
Journal sequence: 0x00000001
Journal start: 0
Group 0: (Blocks 1-8192)
Primary superblock at 1, Group descriptors at 2-2
Reserved GDT blocks at 3-129
Block bitmap at 130 (+129)
Inode bitmap at 132 (+131)
Inode table at 134-389 (+133)
7495 free blocks, 491 free inodes, 5 directories, 491 unused inodes
Free blocks: 698-8192
Free inodes: 22-512
Group 1: (Blocks 8193-16383) [INODE_UNINIT]
Backup superblock at 8193, Group descriptors at 8194-8194
Reserved GDT blocks at 8195-8321
Block bitmap at 131 (bg #0 + 130)
Inode bitmap at 133 (bg #0 + 132)
Inode table at 390-645 (bg #0 + 389)
7038 free blocks, 512 free inodes, 0 directories, 512 unused inodes
Free blocks: 9346-16383
Free inodes: 513-1024
debugfs: stat /emptyfile
Inode: III Type: regular
Size: 0
Fragment: Address: 0 Number: 0 Size: 0
debugfs: stat /bigfile
Inode: III Type: regular
Size: 32768
Fragment: Address: 0 Number: 0 Size: 0
debugfs: stat /sparsefile
Inode: III Type: regular
Size: 1073741825
Fragment: Address: 0 Number: 0 Size: 0
debugfs: stat /bigzerofile
Inode: III Type: regular
Size: 1073741825
Fragment: Address: 0 Number: 0 Size: 0
debugfs: stat /fifo
debugfs: stat /emptydir
Inode: III Type: directory
Size: 60
Fragment: Address: 0 Number: 0 Size: 0
debugfs: stat /dir
Inode: III Type: directory
Size: 60
Fragment: Address: 0 Number: 0 Size: 0
debugfs: stat /dir/file
Inode: III Type: regular
Size: 8
Fragment: Address: 0 Number: 0 Size: 0
debugfs: stat /acl_dir
Inode: III Type: directory
Size: 60
Fragment: Address: 0 Number: 0 Size: 0
debugfs: stat /acl_dir/file
Inode: III Type: regular
Size: 10
Fragment: Address: 0 Number: 0 Size: 0
debugfs: ea_list dir/file
Extended attributes:
system.data (0)
debugfs: ea_list acl_dir
Extended attributes:
system.data (0)
system.posix_acl_access (28) = 01 00 00 00 01 00 07 00 04 00 05 00 08 00 05 00 2a 00 00 00 10 00 05 00 20 00 05 00
system.posix_acl_default (28) = 01 00 00 00 01 00 07 00 04 00 05 00 08 00 05 00 04 00 00 00 10 00 05 00 20 00 05 00
debugfs: ea_list acl_dir/file
Extended attributes:
system.data (0)
system.posix_acl_access (28) = 01 00 00 00 01 00 07 00 04 00 05 00 08 00 05 00 2a 00 00 00 10 00 05 00 20 00 05 00
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
test.img: 21/1024 files (0.0% non-contiguous), 1851/16384 blocks
Apart from 32bit ppc instead of 64bit ppc another difference is the underlying root filesystem. On ppc64 I use btrfs (rw,relatime,compress=lzo,ssd,discard=async,space_cache=v2,subvolid=5,subvol=/)
on ppc I use ext4 (rw,relatime)
. Could this make a difference?
Another difference is using GCC 11.4 on ppc vs. using GCC 12.3 on ppc64. I'll check out next whether I get different results using GCC 11.4 on ppc64.
By the way, I don't know how long I will have access to a ppc64 Big Endian machine.
Perhaps the GCC Compile Farm may be of interest for you? They got one POWER7 and one POWER8 machine running ppc64 BE (and several others ppc64le) where you are granted free access when working on a piece of free software (GCC or any other GPL, BSD, MIT, ...) and need ssh access to the farm for compilation, debug and test on various architectures.
Apart from that I tested out my assumptions from my last post. GCC 11.4 vs. GCC 12.4 does not matter but underlying filesystem does!
When I build e2fsprogs and run the tests from an ext4 partition m_rootdir_acl passes - when I build e2fsprogs and run the tests from a btrfs partition m_rootdir_acl fails.
Question is this being an e2fsprogs test-suite bug or a btrfs bug?
The way this test works is to set up extended attributes on the local directory where the test is being run, and then creates a file system with those extended attributes using "mke2fs -d". What this is testing is how mke2fs reads the extended attributes from the local file system, and populates the freshly created file system with those extended attributes.
Running on xfs, the test passes:
% /usr/projects/e2fsprogs/e2fsprogs-maint/configure ; make -j16 ; make -j16 check
...
% cd tests
% ./test_script m_rootdir_acl
m_rootdir_acl: create fs image from dir using inline_data and acls: ok
373 tests succeeded 0 tests failed
% df .
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/loop1 983040 106224 876816 11% /mnt
% blkid /dev/loop1
/dev/loop1: UUID="528cda99-0c16-433c-bcea-55938bd63850" BLOCK_SIZE="512" TYPE="xfs"
Running on btrfs, it looks like the problem is the order that it is returning the extended attributes when read from the local file system is different from ext4 and xfs:
% rm -f /tmp/foo.img ; truncate -s 1G /tmp/foo.img ; mkfs.btrfs /tmp/foo.img
% sudo mount /tmp/foo.img /mnt ; sudo chown tytso /mnt; cd /mnt
% /usr/projects/e2fsprogs/e2fsprogs-maint/configure ; make -j16 ; make -j16 check
% cat tests/m_rootdir_acl.failed
--- /usr/projects/e2fsprogs/e2fsprogs-maint/tests/m_rootdir_acl/expect 2023-02-05 20:16:43.407233029 +
0000
+++ m_rootdir_acl.log 2023-08-09 15:18:15.681548566 +0000
@@ -105,8 +105,8 @@
debugfs: ea_list acl_dir
Extended attributes:
system.data (0)
- system.posix_acl_access (28) = 01 00 00 00 01 00 07 00 04 00 05 00 08 00 05 00 2a 00 00 00 10 00 05
00 20 00 05 00
system.posix_acl_default (28) = 01 00 00 00 01 00 07 00 04 00 05 00 08 00 05 00 04 00 00 00 10 00 05
00 20 00 05 00
+ system.posix_acl_access (28) = 01 00 00 00 01 00 07 00 04 00 05 00 08 00 05 00 2a 00 00 00 10 00 05
00 20 00 05 00
debugfs: ea_list acl_dir/file
Extended attributes:
system.data (0)
I ran the above reproduction on x86, so this is not a ppc64 specific issue. ppc32 and ppc64 was a red-herring; it has nothing to do with big-endian systems, but rather how the file system returns the order of the xattrs. The main issue is that most of the ext4 development communities tend to either use ext4 or xfs file systems. Or if they run the regression test suite on MacOS, or on Linux with tmpfs, the test gets skipped because MacOS doesn't support extended attributes, and tmpfs doesn't support user extended attributes.
@tytso Should I open an issue on the btrfs tracker regarding this issue here?
Well, actually, I don't think there is any guarantee of ordering with respect to the ordering of how extended attributes are returned --- certainly not between system.posix_acl_default and system.posix_acl_access! Ext4 will certainly move where extended attributes are stored between in the inode (where it is faster, for things like ACL's and SELinux labels, where that is critcal), and in an external xattr block, so the ordering of how ext4 store extended attributes is also subject to change.
So this is arguably a test bug, where we need to sort the order of extended attributes returned by dumpe2fs to see whether or not the xattrs were accurately placed in the file system. It's a non-trivial change, though so it's one of those things that I probably won't have time to fix in the immediate future. Patches would greatfully be accepted....
Thanks for clarification!
This started with Gentoo downstream bug #905221.
I wanted to bisect the change leading to the _m_rootdiracl failure but found out the test was failing since its' introduction in v1.46.1. Test still fails with git-master.
m_rootdir_acl.failed:
m_rootdir_acl.log:
Tests were run from git-master on a Talos II running Gentoo Linux, Big Endian. EXT4_FS_POSIX_ACL=y is enabled in the kernel .config, ext4 is built into the kernel (not as a module).
testlog_git-master.log