rincebrain
commented
1 year ago That stack trace is you triggering an OOM from the ext4 code, not a kernel panic.
ZFS's memory usage there doesn't seem absurd, from the output, but there seems to be a lot of RAM not covered by the output there if it's got 32 GIB in total.
Could you share more of the output, ideally the first 200 or so lines after it starts to go off the rails?
e: also cat /proc/slabinfo over time would be good to capture. My guess here would be, since it's reporting a lot of memory usage in unreclaimable slabs but the math doesn't seem to add up there, assuming I'm not missing a zero somewhere, that memory fragmentation is biting you, and you're making and (sometimes) freeing lots of small allocations, then when you actually want a contiguous 1M chunk (or some similarly large one) for something, and it's an allocation that can't fail, we're now running around killing things trying to free enough, but it still isn't winding up finding a large enough contiguous chunk anywhere.
In theory, I think the OOM messages should print somewhere around when they start how much they're trying to find, so that'd be a hint.
Frank030366
tmcqueen-materials
System information
Describe the problem you're observing
System getting kernel panic everytime when running simple bonnie++ test on ZFS filesystem (not zvol) with recordsize set to 1M (512K affected as well). If setting recordsize to 256K or lower test is running fine. This test was performed on bare metal hardware (Intel i3 12 gen, 32GB RAM), but I was able to reproduce it on VMware virtual machine as well.
Describe how to reproduce the problem
apt-get update && apt-get -y dist-upgrade);/etc/modprobe.d/zfs.conf;zpool create -o ashift=12 testpool raidz2 /dev/disk/by-id/<disk1_name> /dev/disk/by-id/<disk8_name>;zfs set atime=off relatime=on xattr=sa dnodesize=auto logbias=throughput sync=standard dedup=off recordsize=1M compression=off testpool;useradd frank && cd /testpool && mkdir bon && chmod 777 bon;bonnie++ -d /testpool/bon/ -x 1 -u frank.Include any warning/errors/backtraces from the system logs
ZFS pool config (last characters obfuscated):
Disk info (all 8 HDD are idetical):
Dmesg trace (last 180 lines):