search

zfsonlinux / grub

GRUB enhancements for ZFS on Linux
16 stars 18 forks source link

grub-probe error: unknown filesystem #19

Open azeemism opened 9 years ago

azeemism commented 9 years ago

Hi,

Regarding the dailies version of zfs: 0.6.3-22~2d9d57~jessie

Mounted zfs files systems are not detected by grub-probe with "error: unknown filesystem" being returned. No such issue found with version v0.6.3-766_gfde0d6d (non-dailies). I am running Debian Jessie of VirtualBox.

All packages below are version: 2.02~beta2-19 amd64 grub-efi grub-efi-amd64 grub-efi-amd64-bin grub2-common

Thanks for all that you do,

Azeem

FransUrbo commented 9 years ago

Mounted zfs files systems are not detected by grub-probe with "error: unknown filesystem" being returned.

All packages below are version: 2.02~beta2-19 amd64 grub-efi grub-efi-amd64 grub-efi-amd64-bin grub2-common

Since you're using efi, I have no idea how to proceed, never tried it.

azeemism commented 9 years ago

To duplicate the issue:

Install Jessie in VirtualBox (tested in RAID 10) sda1 - UEFI partition (must be on sda1 or sdb1 only for VirtualBox) sd[a-d]2 - unformatted partition for zfs sd[a-d]3 - swap on sd[a-d]4 - /boot on ext4 RAID 10 sd[a-d]5 - / on ext4 RAID 10

EFI will already be install by select the option in Debian Jessie, Install zfs dailies, zpool create -m none spool mirror /dev/disk/by-id /dev/disk/by-id mirror /dev/disk/by-id /dev/disk/by-id create datasets for spool/ROOT/debian zfs set mountpoint=/test spool/ROOT/debian Fix the canonical path error: #5

cat > /etc/udev/rules.d/70-zfs-grub-fix.rules << 'EOF' ENV{DEVTYPE}=="partition", IMPORT{parent}="ID_*", ENV{ID_FS_TYPE}=="zfs_member", SYMLINK+="$env{ID_BUS}-$env{ID_SERIAL} $env{ID_BUS}-$env{ID_SERIAL}-part%n" EOF udevadm trigger

reboot modprobe efivars <-- should not be needed but can run it just in case grub-probe /test

Since "/" is not on zfs at boot time, would it matter if it is efi?

FransUrbo commented 9 years ago

zfs set mountpoint=/test spool/ROOT/debian

That's wrong. It should be 'mountpoint=/'. Since "/" is not on zfs at boot time, would it matter if it is efi?

No idea, but I know that EFI is somewhat problematic when it comes to grub. I once tried to get a 'normal' system running on EFI but failed...

azeemism commented 9 years ago

That's wrong. It should be 'mountpoint=/'.

Yes you are right. I was running a quick test with grub-probe and only one folder. It should have been zfs set mountpoint=/test spool/ROOT/debian/test.

No idea, but I know that EFI is somewhat problematic when it comes to grub. I once tried to get a 'normal' system running on EFI but failed...

With the current Debian version of grub-efi-amd64 I can confirm success with the zfs non-daily.

I think more people are having success these days, but there isn't a clear document on the web outlining each step of the process to achieve the goal. To be fair to yourself, in the past it was a lot more difficult and they were many more bugs to work through that perhaps made it impossible/impractical in many situations.

A lot of progress has been made that perhaps now allows for a different perspective other than the horizon approach where the goal is still out of reach.

On a more tangible note, my next comment may have some clues at to why this issue is occurring.

azeemism commented 9 years ago

I created a pool with only one dataset mounted:

zfs create -o mountpoint=/stest spool/ROOT/debian/stest

I then use grub probe's verbose output (version: 2.02~beta2-10)

grub-probe -vvv /stest

For both the, dailies: where zfs is not detected non-daily: where zfs is detected

They are some difference between the two outputs that I hope will help to sort out the current issue.

What stood out to me is that at an initial level grub-probe detects zfs on the dailies before there seems to be a failure:

grub-core/fs/zfs/zfs.c:1107: check 10 passed
grub-core/fs/zfs/zfs.c:1123: str=com.delphix:hole_birth
grub-core/fs/zfs/zfs.c:1126: feature missing in check_pool_label:com.delphix:hole_birth
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/fs/zfs/zfs.c:1187: label ok 2
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/fs/zfs/zfs.c:1187: label ok 3
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/kern/fs.c:78: zfs detection failed.

_Is there a mismatch in the order or sequence of features (between the dailies and non-daily) that is causing grub-probe to fail?_

dailies

non-daily

Full Output of Dailies v0.6.3-24~6466b6~jessie

grub-core/kern/disk.c:196: Opening `hostdisk//dev/sde'...
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sde'...
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sde' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde'
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sde1' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sde9' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde9'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde9'
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sde'.
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sde'.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdf'...
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdf'...
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sdf' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf'
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sdf1' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf1'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf1'
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sdf9' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf9'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf9'
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdf'.
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdf'.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sde'...
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sde'...
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sde' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde'
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sde'.
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sde'.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sde'...
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sde'...
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sde' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde'
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sde'.
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sde'.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdf'...
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdf'...
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sdf' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf'
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdf'.
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdf'.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdf'...
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdf'...
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sdf' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf'
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdf'.
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdf'.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sde,gpt1'...
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/kern/fs.c:56: Detecting zfs...
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sde1' in open_device()
grub-core/fs/zfs/zfs.c:1187: label ok 0
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/fs/zfs/zfs.c:1003: check 2 passed
grub-core/fs/zfs/zfs.c:1014: check 3 passed
grub-core/fs/zfs/zfs.c:1021: check 4 passed
grub-core/fs/zfs/zfs.c:1031: check 6 passed
grub-core/fs/zfs/zfs.c:1039: check 7 passed
grub-core/fs/zfs/zfs.c:1050: check 8 passed
grub-core/fs/zfs/zfs.c:1060: check 9 passed
grub-core/fs/zfs/zfs.c:1082: check 11 passed
grub-core/fs/zfs/zfs.c:1107: check 10 passed
grub-core/fs/zfs/zfs.c:1123: str=com.delphix:hole_birth
grub-core/fs/zfs/zfs.c:1126: feature missing in check_pool_label:com.delphix:hole_birth
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/fs/zfs/zfs.c:1187: label ok 1
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/fs/zfs/zfs.c:1003: check 2 passed
grub-core/fs/zfs/zfs.c:1014: check 3 passed
grub-core/fs/zfs/zfs.c:1021: check 4 passed
grub-core/fs/zfs/zfs.c:1031: check 6 passed
grub-core/fs/zfs/zfs.c:1039: check 7 passed
grub-core/fs/zfs/zfs.c:1050: check 8 passed
grub-core/fs/zfs/zfs.c:1060: check 9 passed
grub-core/fs/zfs/zfs.c:1082: check 11 passed
grub-core/fs/zfs/zfs.c:1107: check 10 passed
grub-core/fs/zfs/zfs.c:1123: str=com.delphix:hole_birth
grub-core/fs/zfs/zfs.c:1126: feature missing in check_pool_label:com.delphix:hole_birth
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/fs/zfs/zfs.c:1187: label ok 2
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/fs/zfs/zfs.c:1187: label ok 3
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/kern/fs.c:78: zfs detection failed.
grub-core/kern/fs.c:56: Detecting xfs...
grub-core/kern/fs.c:78: xfs detection failed.
grub-core/kern/fs.c:56: Detecting ufs2...
grub-core/kern/fs.c:78: ufs2 detection failed.
grub-core/kern/fs.c:56: Detecting ufs1_be...
grub-core/kern/fs.c:78: ufs1_be detection failed.
grub-core/kern/fs.c:56: Detecting ufs1...
grub-core/kern/fs.c:78: ufs1 detection failed.
grub-core/kern/fs.c:56: Detecting udf...
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/kern/fs.c:78: udf detection failed.
grub-core/kern/fs.c:56: Detecting tarfs...
grub-core/kern/fs.c:78: tarfs detection failed.
grub-core/kern/fs.c:56: Detecting squash4...
grub-core/kern/fs.c:78: squash4 detection failed.
grub-core/kern/fs.c:56: Detecting sfs...
grub-core/kern/fs.c:78: sfs detection failed.
grub-core/kern/fs.c:56: Detecting romfs...
grub-core/kern/fs.c:78: romfs detection failed.
grub-core/kern/fs.c:56: Detecting reiserfs...
grub-core/kern/fs.c:78: reiserfs detection failed.
grub-core/kern/fs.c:56: Detecting procfs...
grub-core/kern/fs.c:78: procfs detection failed.
grub-core/kern/fs.c:56: Detecting odc...
grub-core/kern/fs.c:78: odc detection failed.
grub-core/kern/fs.c:56: Detecting ntfs...
grub-core/kern/fs.c:78: ntfs detection failed.
grub-core/kern/fs.c:56: Detecting nilfs2...
grub-core/kern/fs.c:78: nilfs2 detection failed.
grub-core/kern/fs.c:56: Detecting newc...
grub-core/kern/fs.c:78: newc detection failed.
grub-core/kern/fs.c:56: Detecting minix_be...
grub-core/kern/fs.c:78: minix_be detection failed.
grub-core/kern/fs.c:56: Detecting minix3_be...
grub-core/kern/fs.c:78: minix3_be detection failed.
grub-core/kern/fs.c:56: Detecting minix3...
grub-core/kern/fs.c:78: minix3 detection failed.
grub-core/kern/fs.c:56: Detecting minix2_be...
grub-core/kern/fs.c:78: minix2_be detection failed.
grub-core/kern/fs.c:56: Detecting minix2...
grub-core/kern/fs.c:78: minix2 detection failed.
grub-core/kern/fs.c:56: Detecting minix...
grub-core/kern/fs.c:78: minix detection failed.
grub-core/kern/fs.c:56: Detecting jfs...
grub-core/kern/fs.c:78: jfs detection failed.
grub-core/kern/fs.c:56: Detecting iso9660...
grub-core/kern/fs.c:78: iso9660 detection failed.
grub-core/kern/fs.c:56: Detecting hfsplus...
grub-core/kern/fs.c:78: hfsplus detection failed.
grub-core/kern/fs.c:56: Detecting hfs...
grub-core/kern/fs.c:78: hfs detection failed.
grub-core/kern/fs.c:56: Detecting fat...
grub-core/kern/fs.c:78: fat detection failed.
grub-core/kern/fs.c:56: Detecting ext2...
grub-core/kern/fs.c:78: ext2 detection failed.
grub-core/kern/fs.c:56: Detecting exfat...
grub-core/kern/fs.c:78: exfat detection failed.
grub-core/kern/fs.c:56: Detecting cpiofs_be...
grub-core/kern/fs.c:78: cpiofs_be detection failed.
grub-core/kern/fs.c:56: Detecting cpiofs...
grub-core/kern/fs.c:78: cpiofs detection failed.
grub-core/kern/fs.c:56: Detecting cbfs...
grub-core/kern/disk.c:421: Read out of range: sector 0x100217ebe (attempt to read or write outside of partition).
grub-core/kern/fs.c:78: cbfs detection failed.
grub-core/kern/fs.c:56: Detecting btrfs...
grub-core/kern/fs.c:78: btrfs detection failed.
grub-core/kern/fs.c:56: Detecting bfs...
grub-core/kern/fs.c:78: bfs detection failed.
grub-core/kern/fs.c:56: Detecting afs...
grub-core/kern/fs.c:78: afs detection failed.
grub-core/kern/fs.c:56: Detecting affs...
grub-core/kern/fs.c:78: affs detection failed.

Full output of Non-Daily v0.6.3-766_gfde0d6d

grub-core/kern/disk.c:196: Opening `hostdisk//dev/sde'...
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sde'...
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sde' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde'
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sde1' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sde9' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde9'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde9'
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sde'.
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sde'.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdf'...
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdf'...
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sdf' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf'
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sdf1' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf1'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf1'
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sdf9' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf9'
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf9'
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdf'.
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdf'.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sde'...
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sde'...
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sde' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde'
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sde'.
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sde'.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sde'...
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sde'...
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sde' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde'
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sde
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sde'.
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sde'.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdf'...
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdf'...
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sdf' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf'
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdf'.
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdf'.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdf'...
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdf'...
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sdf' in open_device()
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sdf'
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:114: GPT entry 8: start=4294946816, length=16384
grub-core/disk/diskfilter.c:135: Scanning for DISKFILTER devices on disk hostdisk//dev/sdf
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdf'.
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdf'.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sde,gpt1'...
grub-core/partmap/gpt.c:90: Read a valid GPT header
grub-core/partmap/gpt.c:114: GPT entry 0: start=2048, length=4294944768
grub-core/kern/fs.c:56: Detecting zfs...
grub-core/osdep/hostdisk.c:415: opening the device `/dev/sde1' in open_device()
grub-core/fs/zfs/zfs.c:1187: label ok 0
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/fs/zfs/zfs.c:1003: check 2 passed
grub-core/fs/zfs/zfs.c:1014: check 3 passed
grub-core/fs/zfs/zfs.c:1021: check 4 passed
grub-core/fs/zfs/zfs.c:1031: check 6 passed
grub-core/fs/zfs/zfs.c:1039: check 7 passed
grub-core/fs/zfs/zfs.c:1050: check 8 passed
grub-core/fs/zfs/zfs.c:1060: check 9 passed
grub-core/fs/zfs/zfs.c:1082: check 11 passed
grub-core/fs/zfs/zfs.c:1107: check 10 passed
grub-core/fs/zfs/zfs.c:1132: check 12 passed (feature flags)
grub-core/fs/zfs/zfs.c:1846: endian = -1
grub-core/fs/zfs/zfs.c:591: dva=8, 190
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/fs/zfs/zfs.c:2616: endian = -1, blkid=0
grub-core/fs/zfs/zfs.c:1960: endian = -1
grub-core/fs/zfs/zfs.c:1986: endian = -1
grub-core/fs/zfs/zfs.c:1846: endian = -1
grub-core/fs/zfs/zfs.c:591: dva=8, 188
grub-core/fs/zfs/zfs.c:2620: alive
grub-core/fs/zfs/zfs.c:2431: looking for 'features_for_read'
grub-core/fs/zfs/zfs.c:1960: endian = 1
grub-core/fs/zfs/zfs.c:1986: endian = 1
grub-core/fs/zfs/zfs.c:1846: endian = 1
grub-core/fs/zfs/zfs.c:591: dva=8, 838
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/fs/zfs/zfs.c:2441: zap read
grub-core/fs/zfs/zfs.c:2445: micro zap
grub-core/fs/zfs/zfs.c:2448: returned 0
grub-core/fs/zfs/zfs.c:1960: endian = 1
grub-core/fs/zfs/zfs.c:1986: endian = 1
grub-core/fs/zfs/zfs.c:1846: endian = 1
grub-core/fs/zfs/zfs.c:591: dva=8, 588
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/fs/zfs/zfs.c:2047: zap: name = org.illumos:lz4_compress, value = 1, cd = 0
grub-core/fs/zfs/zfs.c:2047: zap: name = , value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2047: zap: name = , value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2047: zap: name = , value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2047: zap: name = , value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2047: zap: name = , value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2047: zap: name = , value = 0, cd = 0
grub-core/fs/zfs/zfs.c:3188: alive
grub-core/fs/zfs/zfs.c:3000: endian = 1
grub-core/fs/zfs/zfs.c:2616: endian = 1, blkid=0
grub-core/fs/zfs/zfs.c:1960: endian = 1
grub-core/fs/zfs/zfs.c:1986: endian = 1
grub-core/fs/zfs/zfs.c:1846: endian = 1
grub-core/fs/zfs/zfs.c:591: dva=8, 188
grub-core/fs/zfs/zfs.c:2620: alive
grub-core/fs/zfs/zfs.c:3007: alive
grub-core/fs/zfs/zfs.c:2431: looking for 'root_dataset'
grub-core/fs/zfs/zfs.c:1960: endian = 1
grub-core/fs/zfs/zfs.c:1986: endian = 1
grub-core/fs/zfs/zfs.c:1846: endian = 1
grub-core/fs/zfs/zfs.c:591: dva=8, 838
grub-core/fs/zfs/zfs.c:2441: zap read
grub-core/fs/zfs/zfs.c:2445: micro zap
grub-core/fs/zfs/zfs.c:2448: returned 0
grub-core/fs/zfs/zfs.c:3013: alive
grub-core/fs/zfs/zfs.c:3019: alive
grub-core/fs/zfs/zfs.c:3197: alive
grub-core/fs/zfs/zfs.c:3201: endian = 0
grub-core/fs/zfs/zfs.c:3211: endian = 1
grub-core/fs/zfs/zfs.c:3065: endian = 1
grub-core/fs/zfs/zfs.c:1846: endian = 1
grub-core/fs/zfs/zfs.c:591: dva=8, 368
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/fs/zfs/zfs.c:3291: endian = 1
grub-core/fs/zfs/zfs.c:3065: endian = 1
grub-core/fs/zfs/zfs.c:1846: endian = 1
grub-core/fs/zfs/zfs.c:591: dva=8, 368
grub-core/fs/zfs/zfs.c:2616: endian = 1, blkid=0
grub-core/fs/zfs/zfs.c:1960: endian = 1
grub-core/fs/zfs/zfs.c:1991: endian = 1
grub-core/fs/zfs/zfs.c:1846: endian = 1
grub-core/fs/zfs/zfs.c:591: dva=8, 348
grub-core/fs/zfs/zfs.c:1960: endian = 1
grub-core/fs/zfs/zfs.c:1991: endian = 1
grub-core/fs/zfs/zfs.c:1846: endian = 1
grub-core/fs/zfs/zfs.c:591: dva=8, 340
grub-core/fs/zfs/zfs.c:1960: endian = 1
grub-core/fs/zfs/zfs.c:1991: endian = 1
grub-core/fs/zfs/zfs.c:1846: endian = 1
grub-core/fs/zfs/zfs.c:591: dva=8, 338
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/fs/zfs/zfs.c:1960: endian = 1
grub-core/fs/zfs/zfs.c:1991: endian = 1
grub-core/fs/zfs/zfs.c:1846: endian = 1
grub-core/fs/zfs/zfs.c:591: dva=8, 330
grub-core/fs/zfs/zfs.c:1960: endian = 1
grub-core/fs/zfs/zfs.c:1991: endian = 1
grub-core/fs/zfs/zfs.c:1846: endian = 1
grub-core/fs/zfs/zfs.c:591: dva=8, 328
grub-core/fs/zfs/zfs.c:1960: endian = 1
grub-core/fs/zfs/zfs.c:1991: endian = 1
grub-core/fs/zfs/zfs.c:1846: endian = 1
grub-core/fs/zfs/zfs.c:591: dva=8, 320
grub-core/fs/zfs/zfs.c:1960: endian = 1
grub-core/fs/zfs/zfs.c:1986: endian = 1
grub-core/fs/zfs/zfs.c:1846: endian = 1
grub-core/fs/zfs/zfs.c:591: dva=8, 318
grub-core/fs/zfs/zfs.c:2620: alive
grub-core/fs/zfs/zfs.c:2431: looking for 'ROOT'
grub-core/fs/zfs/zfs.c:1960: endian = 1
grub-core/fs/zfs/zfs.c:1986: endian = 1
grub-core/fs/zfs/zfs.c:1846: endian = 1
grub-core/fs/zfs/zfs.c:591: dva=8, 0
grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sde1'
grub-core/fs/zfs/zfs.c:2441: zap read
grub-core/fs/zfs/zfs.c:2445: micro zap
grub-core/fs/zfs/zfs.c:2448: returned 0
zfs
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sde'.

End Remarks

I hope this helps shed some light that can lead to a fix.

There was a post for Ubuntu

https://github.com/zfsonlinux/pkg-zfs/issues/75

That suggested creating a pool as follows,

zpool create -o version=28 -o ashift=9 tank ...

But this is not the case with Debian Jessie, as the latest version 5000 and either of ashift=12 or ashift=13 do work with the non-daily version.

non-daily-2

FransUrbo commented 9 years ago

I create a pool with only one dataset mounted: zfs create -o mountpoint=/stest spool/ROOT/debian/stest

[…]

Could you edit that post, and use the 'triple back ticks markdown' to distinguish the code segments. The post is very long and very difficult to read/overview as it is now.

See https://help.github.com/articles/github-flavored-markdown/#syntax-highlighting

wallzero commented 9 years ago

I believe I have the same issue. I have been running a ZFS root installation with Debian Jessie for months with no issue. Then, I installed the nightly builds, and now I am unable to upgrade the kernel. I receive the following error:

dpkg: error processing package linux-image-3.16.0-4-amd64 (--configure):
 subprocess installed post-installation script returned error exit status 2
Setting up grub-efi-amd64 (2.02~beta2-20) ...
/usr/sbin/grub-probe: error: cannot find a device for / (is /dev mounted?).
azeemism commented 9 years ago

@FransUrbo I couple days again, I created bug reports with Debian as I had mentioned I would.

deb bugs

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=776676 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=777191

This also addresses:

18

https://github.com/zfsonlinux/zfs/issues/3047

Debian Bug 777191 requests that they look into applying the following patches to upstream grub-efi-amd64:

LIST OF PATCHES:

_https://github.com/zfsonlinux/grub/tree/snapshot/debian/wheezy/2.02_beta2%2Bgitbac5d-9.ZOL1/debian/patches_

    blacklist_1440x900x32.patch
    core_in_fs.patch
    default_grub_d.patch
    disable_floppies.patch
    dpkg_version_comparison.patch
    elf_bi_endian.patch
    feature_flags.patch
    first_device.patch
    fix_variable_redeclarations.patch
    gettext_quiet.patch
    gfxpayload_dynamic.patch
    gfxpayload_keep_default.patch
    grub-shell-no-pad.patch
    grub-tester-zfs.patch
    grub.cfg_400.patch
    grub_legacy_0_based_partitions.patch
    ignore_grub_func_test_failures.patch
    install_efi_fallback.patch
    install_locale_langpack.patch
    install_powerpc_machtypes.patch
    install_stage2_confusion.patch
    linuxefi.patch
    linuxefi_amd64_only.patch
    linuxefi_debug.patch
    linuxefi_non_sb_fallback.patch
    linuxefi_require_shim.patch
    maybe_quiet.patch
    mkconfig_loopback.patch
    mkconfig_mid_upgrade.patch
    mkconfig_nonexistent_loopback.patch
    mkconfig_recovery_title.patch
    mkconfig_signed_kernel.patch
    mkconfig_ubuntu_distributor.patch
    mkconfig_ubuntu_recovery.patch
    multiple-raidz-device-detection.txt
    no_insmod_on_sb.patch
    no_redefine_MIN-MAX.patch
    probe_fusionio.patch
    quick_boot.patch
    restore_mkdevicemap.patch
    series  * Add support for spacemap_histogram, enabled_txg, extensible_dataset.
    skip_gettext_strings_test.patch
    sleep_shift.patch
    uefi_firmware_setup.patch
    vt_handoff.patch
    wubi_no_windows.patch
toobuntu commented 9 years ago

See Debian Bug#775395 - partman-zfs in d-i jessie image does not create grub-compatible /boot ZFS mirror

Workaround:

What I ended up doing is recreating everything again, this time interrupting and adding the "-d" flag to the pool creation command (done manually) to leave all the feature@* options disabled for building the system.

This worked. Installation of Grub as the boot loader was successful and the system booted using /boot (and /lib/modules) as volumes on the "b" pool, which is a simple ZFS mirror.

Please consider adding the "-d" flag in partman-zfs for Jessie, i.e.,

# zpool create -d "pool" <dev-spec>

And then perhaps turning on feature@ flags that are known to work with grub for booting.

azeemism commented 9 years ago

@toobuntu I was able to install with no issues using,

ZoL version v0.6.3-766_gfde0d6d (non-dailies) Debian Jessie weekly build: http://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/weekly-builds/amd64/iso-cd/

grub-efi-amd64 version: 2.02~beta2-19 amd64, or 2.02~beta2-20 amd64

Even had /boot under ZFS.

Pool was created with,

zpool create -m none -o ashift=13 -o feature@lz4_compress=enabled -o autoreplace=on -o autoexpand=on spool mirror /dev/disk/by-id/scsi-SATA_disk1-part2 /dev/disk/by-id/scsi-SATA_disk2-part2 mirror /dev/disk/by-id/scsi-SATA_disk3-part2 /dev/disk/by-id/scsi-SATA_disk4-part2

The same success was not achieved with the ZoL dailies version where I simplified pool creation to

zpool create -m none spool mirror /dev/disk/by-id/scsi-SATA_disk1-part2 /dev/disk/by-id/scsi-SATA_disk2-part2 mirror /dev/disk/by-id/scsi-SATA_disk3-part2 /dev/disk/by-id/scsi-SATA_disk4-part2

I have not tested with the -d option.

But installing Grub failed with "unknown filesystem". In troubleshooting the issue, it seems grub2 (version currently is 2.02~beta2-19) does not recognize the "feature@lz4_compress" option

I agree there is an issue with grub, but there seems to be more to it? as i got it working with LZ4 and the non-daily ZoL code.

Please consider adding the "-d" flag in partman-zfs for Jessie

I would rather a fix be implemented so grub2 and ZoL can work with all the features. I certainly don't want to lose out on LZ4.

Thanks for pointing out your findings. I will link it to my Debian Bug Report.

wallzero commented 9 years ago

A recent upgrade caused an unsuccessful boot. I chrooted into my system and I upgraded to a grub version from the experimental branch. Within the chroot, initramfs completed successfully. I then tried to boot again but I was stalled in busybox. I use the following commands to continue:

zpool import rpool -R /root
exit
exit

The first exit returns an error the original error but the second proceeds to boot successfully. I can now update initramfs within my install, but still get stuck in busybox. This is my /etc/default/grub file:

# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash init=/bin/systemd"
GRUB_CMDLINE_LINUX="boot=zfs rpool=rpool rcutree.rcu_idle_gp_delay=1"

# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"

# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
GRUB_GFXMODE=1600x900

# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY="true"

# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"
FransUrbo commented 9 years ago

A recent upgrade caused an unsuccessful boot. I chrooted into my system and I upgraded to a grub version from the experimental branch. Within the chroot, initramfs completed successfully. I then tried to boot again but I was stalled in busybox. I use the following commands to continue:

This sounds like a problem with the initrd script, not grub.

Popyllol commented 9 years ago

So... is this the right place to ask when com.delphix:hole_birth will be supported by grub? I am booting another Linux partition until it will be supported, but... would be nice to boot my real-Linux soon again! :-) My problem is this, too: grub-core/fs/zfs/zfs.c:1126: feature missing in check_pool_label:com.delphix:hole_birth Greetings, Lars

FransUrbo commented 9 years ago

So... is this the right place to ask when com.delphix:hole_birth will be supported by grub?

In the Debian GNU/Linux packages, they have for the last two-three months.

Regarding 'all other' distributions, somewhere between 'in a year or two' to 'never'…

The 'problem' is that no-one (except me) is willing to carry the patch(es) that @tsoome did for Illumos. They're CDDL and Grub2 is GPL and they 'clash'. Or at least that's the general opinion. I'm not a lawyer, so I can't say for sure but there you are…

@tsoome is currently in the process of rewriting the patch(es) and pushing it to Grub upstream. But your guess is as good as mine when that will be accepted. Until then, no-one is willing to do the work.

tsoome commented 9 years ago

check out the upstream git on savannah;) only one missing at the moment is extensible dataset feature, its posted but they are probably still testing it, poke grub-devel’s.

rgds, toomas

On 04.05.2015, at 14:20, Turbo Fredriksson notifications@github.com wrote:

So... is this the right place to ask when com.delphix:hole_birth will be supported by grub?

In the Debian GNU/Linux packages, they have for the last two-three months.

Regarding 'all other' distributions, somewhere between 'in a year or two' to 'never'…

The 'problem' is that no-one (except me) is willing to carry the patch(es) that @tsoome did for Illumos. They're CDDL and Grub2 is GPL and they 'clash'. Or at least that's the general opinion. I'm not a lawyer, so I can't say for sure but there you are…

@tsoome is currently in the process of rewriting the patch(es) and pushing it to Grub upstream. But your guess is as good as mine when that will be accepted. Until then, no-one is willing to do the work. — Reply to this email directly or view it on GitHub https://github.com/zfsonlinux/grub/issues/19#issuecomment-98680713.

Popyllol commented 9 years ago

Thanks for your fast response! But... hmm... what did I wrong? I looked in sid for the newest version to test the grub-probe:

# wget http://ftp.de.debian.org/debian/pool/main/g/grub2/grub-common_2.02~beta2-22_amd64.deb
# dpkg-deb --extract grub-common_2.02~beta2-22_amd64.deb /tmp/
# /tmp/usr/sbin/grub-probe /zfs
/tmp/usr/sbin/grub-probe: error: unknown filesystem.
# /tmp/usr/sbin/grub-probe -vvv /zfs 2>&1 | grep birth
grub-core/fs/zfs/zfs.c:1123: str=com.delphix:hole_birth
grub-core/fs/zfs/zfs.c:1126: feature missing in check_pool_label:com.delphix:hole_birth
grub-core/fs/zfs/zfs.c:1123: str=com.delphix:hole_birth
grub-core/fs/zfs/zfs.c:1126: feature missing in check_pool_label:com.delphix:hole_birth

Same effect. Any further hint for me?

FransUrbo commented 9 years ago

Thanks for your fast response! But... hmm... what did I wrong? I looked in sid for the newest version to test the grub-probe:

I said I did, not that Debian GNU/Linux did :).

In my Debian GNU/Linux for ZoL repositories, I have 2.02-beta2.9-ZOL11-7aa9f6-wheezy (and 2.02-beta2.9-ZOL11-7aa9f6 for Jessie) which supports all new features of ZoL.

Popyllol commented 9 years ago

Ahhh... stupid me! Reading the content is better than just reading the letters ;-).

tsoome commented 9 years ago

also note, the upstream grub did apply the commits just few days ago (and as I wrote, it is still missing the extensible dataset one, which is needed for large_block support), so the distro maintainers probably haven’t picked them up yet.

rgds, toomas

On 04.05.2015, at 15:06, Turbo Fredriksson notifications@github.com wrote:

Thanks for your fast response! But... hmm... what did I wrong? I looked in sid for the newest version to test the grub-probe:

I said I did, not that Debian GNU/Linux did :).

In my Debian GNU/Linux for ZoL repositories, I have 2.02-beta2.9-ZOL11-7aa9f6-wheezy (and 2.02-beta2.9-ZOL11-7aa9f6 for Jessie) which supports all new features of ZoL.

— Reply to this email directly or view it on GitHub https://github.com/zfsonlinux/grub/issues/19#issuecomment-98689075.

Popyllol commented 9 years ago

To build the savannah clone was a good hint!

# git clone git://git.savannah.gnu.org/grub.git

I build it from source (yes, I should have packaged it...) and installed it over my grub isntallation... quick and dirty. Then I did grub-install and now I am online again with my Linux on ZFS! Thx to my helpers!

wallzero commented 9 years ago

Just want to chime in again; I no longer have any issues with grub and I'm off daily builds and back to the latest release.

sabey commented 8 years ago

I was stuck on this for 2 days but finally managed to be able to boot into ubuntu 14.04 again, instead of getting stuck in initramfs or something else

I updated/enabled com.delphix:hole_birth and 2.02~beta2-9ubuntu1.7 was no longer able to detect zfs with grub-probe

I ended up building grub from git://git.savannah.gnu.org/grub.git but I ended up getting stuck in initramfs

I edited my new grub.cfg by hand and modified the linux lines (since I don't really know what I'm doing) to use my original grub.cfg and I was able to boot without issue

not working git grub.cfg linux /ROOT/ubuntu-1@/boot/vmlinuz-3.13.0-86-generic root=ZFS=zroot/ROOT/ubuntu-1 ro

original grub cfg: linux /ROOT/ubuntu-1@/boot/vmlinuz-3.13.0-86-generic root=ZFS=/ROOT/ubuntu-1 ro quiet splash boot=zfs rpool=zroot bootfs=zroot/ROOT/ubuntu-1 $vt_handoff

hope this saves someone from a headache

tsoome commented 8 years ago

There you see 2 different issues - one is about old grub in ubuntu not recognizing zfs features, and second one is how (new) grub is generating configuration entry for grub menu. If ubuntu is needing specific parameters for initramfs, you can not just replace grub in place, but need to bring over the configuration generation scripts tailored for the needs of ubuntu. I’d suggest to poke ubuntu devs to update grub in their base - those zfs updates have been in grub for quite some time.

rgds, toomas

On 26. mai 2016, at 9:17, Jackson Sabey notifications@github.com wrote:

I was stuck on this for 2 days but finally managed to be able to boot into ubuntu 14.04 again, instead of getting stuck in initramfs or something else

I updated/enabled com.delphix:hole_birth and 2.02~beta2-9ubuntu1.7 was no longer able to detect zfs with grub-probe

I ended up building grub from git://git.savannah.gnu.org/grub.git but I ended up getting stuck in initramfs

I edited my new grub.cfg by hand and modified the linux lines (since I don't really know what I'm doing) to use my original grub.cfg and I was able to boot without issue

not working git grub.cfg linux /ROOT/ubuntu-1@/boot/vmlinuz-3.13.0-86-generic root=ZFS=zroot/ROOT/ubuntu-1 ro

original grub cfg: linux /ROOT/ubuntu-1@/boot/vmlinuz-3.13.0-86-generic root=ZFS=/ROOT/ubuntu-1 ro quiet splash boot=zfs rpool=zroot bootfs=zroot/ROOT/ubuntu-1 $vt_handoff

hope this saves someone from a headache

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/zfsonlinux/grub/issues/19#issuecomment-221787051

beloncfy commented 7 years ago

Problem persist on Ubuntu 16.04 with all the latest update from official repo as well. It works initially when booting up the server, few days later i got this.

grub-probe / grub-probe: error: unknown filesystem.

gcs-github commented 7 years ago

I'm now running into something similar on gentoo, ZFS 0.6.5.8, with both grub git HEAD and grub-2.02_beta3-r1

This didn't use to happen, even on the same system. I'm wondering if a recently activated feature on my pool could be causing this.

If it helps:

hdpool  feature@async_destroy       enabled                     local
hdpool  feature@empty_bpobj         active                      local
hdpool  feature@lz4_compress        active                      local
hdpool  feature@spacemap_histogram  active                      local
hdpool  feature@enabled_txg         active                      local
hdpool  feature@hole_birth          active                      local
hdpool  feature@extensible_dataset  enabled                     local
hdpool  feature@embedded_data       active                      local
hdpool  feature@bookmarks           enabled                     local
hdpool  feature@filesystem_limits   enabled                     local
hdpool  feature@large_blocks        enabled                     local
bunder2015 commented 7 years ago

It is my understanding that grub doesn't support feature@large_blocks, however that shouldn't cause this error, but something along the lines of "error: filesystem 'zfs' doesn't support blocklists." (I could be wrong on this)

However, is this system gentoo? If you get "unknown filesystem" it might be possible that you didn't enable the libzfs use flag.

I would possibly recommend making a new bug since this one is quite old, and also from a debian/ubuntu system.

gcs-github commented 7 years ago

libzfs is enabled. It really is a used to work / doesn't work anymore situation without any change in grub itself. That's why my logical interpretation was that an active feature flag may be getting in the way of grub's FS recognition. The pool has evolved. The software? Not so much

gcs-github commented 7 years ago

To be clearer maybe: I only started playing with reinstalling various versions of the grub code after the problem became appearant.

gcs-github commented 7 years ago

Relevant output of grub-probe:

grub-core/fs/zfs/zfs.c:1192: label ok 0
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sda'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sda'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sda'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sda'
grub-core/fs/zfs/zfs.c:1192: label ok 1
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sda'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sda'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sda'
grub-core/fs/zfs/zfs.c:1192: label ok 2
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sda'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sda'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sda'
grub-core/fs/zfs/zfs.c:1192: label ok 3
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sda'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sda'
grub-core/kern/fs.c:78: zfs detection failed.
Spongman commented 7 years ago

I'm seeing this problem on debian 8. update-grub creates unbootable zfs entries.

$ zpool status
  pool: rpool
 state: ONLINE
  scan: none requested
config:

        NAME                                       STATE     READ WRITE CKSUM
        rpool                                      ONLINE       0     0     0
          raidz1-0                                 ONLINE       0     0     0
            ata-VBOX_HARDDISK_VB8f7e2b1d-5d5229ec  ONLINE       0     0     0
            ata-VBOX_HARDDISK_VB653ff503-abdcdff1  ONLINE       0     0     0
            ata-VBOX_HARDDISK_VB54daa735-6ca56ca5  ONLINE       0     0     0
            ata-VBOX_HARDDISK_VB7692cab8-2080dce8  ONLINE       0     0     0
            ata-VBOX_HARDDISK_VB5ec34f65-08a883cb  ONLINE       0     0     0

errors: No known data errors

$ ls -l /dev/disk/by-id/ata-VBOX_HARDDISK_VB8f7e2b1d-5d5229ec
lrwxrwxrwx 1 root root 9 Oct 24 22:25 /dev/disk/by-id/ata-VBOX_HARDDISK_VB8f7e2b1d-5d5229ec -> ../../sdb

$ df /
Filesystem          1K-blocks    Used Available Use% Mounted on
rpool/ROOT/debian-1  13960320 1297152  12663168  10% /

$ grub-probe /
zfs

$ grub-probe --device /dev/disk/by-id/ata-VBOX_HARDDISK_VB8f7e2b1d-5d5229ec
grub-probe: error: unknown filesystem.

$ grub-probe --device /dev/sdb
grub-probe: error: unknown filesystem.

$ grub-probe --device /dev/sdb1
grub-probe: error: unknown filesystem.

$ uname -a
Linux debian 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux

$ grub-probe --version
grub-probe (GRUB) 2.02-beta2.9-ZOL11-7aa9f6

$ zpool get all
NAME   PROPERTY                    VALUE                       SOURCE
rpool  size                        19.9G                       -
rpool  capacity                    7%                          -
rpool  altroot                     -                           default
rpool  health                      ONLINE                      -
rpool  guid                        17391250066009800092        default
rpool  version                     -                           default
rpool  bootfs                      rpool/ROOT/debian-1         local
rpool  delegation                  on                          default
rpool  autoreplace                 off                         default
rpool  cachefile                   -                           default
rpool  failmode                    wait                        default
rpool  listsnapshots               off                         default
rpool  autoexpand                  off                         default
rpool  dedupditto                  0                           default
rpool  dedupratio                  1.00x                       -
rpool  free                        18.3G                       -
rpool  allocated                   1.55G                       -
rpool  readonly                    off                         -
rpool  ashift                      12                          local
rpool  comment                     -                           default
rpool  expandsize                  -                           -
rpool  freeing                     0                           default
rpool  fragmentation               1%                          -
rpool  leaked                      0                           default
rpool  feature@async_destroy       enabled                     local
rpool  feature@empty_bpobj         active                      local
rpool  feature@lz4_compress        active                      local
rpool  feature@spacemap_histogram  active                      local
rpool  feature@enabled_txg         active                      local
rpool  feature@hole_birth          active                      local
rpool  feature@extensible_dataset  enabled                     local
rpool  feature@embedded_data       active                      local
rpool  feature@bookmarks           enabled                     local
rpool  feature@filesystem_limits   enabled                     local
rpool  feature@large_blocks        enabled                     local
tsoome commented 7 years ago

you can add -d -or -v to add more verbosity for grub-probe (I can't remember which switch it was;) - so you can get very detailed output to see which feature is missed from your grub. From presented feature list it seems you just do have too old grub, as all listed features should be supported by grub HEAD over an year now...

Spongman commented 7 years ago

this version of grub-probe is the latest in debian jessie.

$ grub-probe -V
grub-probe (GRUB) 2.02-beta2.9-ZOL11-7aa9f6

$ grub-probe -vvv -d /dev/sdb1
grub-probe: info: cannot open `/boot/grub/device.map': No such file or directory.
grub-probe: info: /dev/sdb1 is not present.
grub-probe: info: Looking for /dev/sdb1.
grub-probe: info: /dev/sdb is a parent of /dev/sdb1.
grub-probe: info: /dev/sdb1 starts from 2048.
grub-probe: info: opening the device hostdisk//dev/sdb.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdb'...
grub-probe: info: drive = 0.
grub-probe: info: the size of hostdisk//dev/sdb is 8388608.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdb'...
grub-probe: info: drive = 0.
grub-probe: info: the size of hostdisk//dev/sdb is 8388608.
grub-core/disk/diskfilter.c:137: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb
grub-probe: info: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid1x devices on disk hostdisk//dev/sdb.
grub-core/osdep/hostdisk.c:416: opening the device `/dev/sdb' in open_device()
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb'
grub-probe: info: Scanning for mdraid09 devices on disk hostdisk//dev/sdb.
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb'
grub-probe: info: Scanning for mdraid09_be devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for dmraid_nv devices on disk hostdisk//dev/sdb.
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb'
grub-probe: info: Scanning for ldm devices on disk hostdisk//dev/sdb.
grub-probe: info: scanning hostdisk//dev/sdb for LDM.
grub-core/partmap/gpt.c:93: Read a valid GPT header
grub-core/partmap/gpt.c:117: GPT entry 0: start=2048, length=8368128
grub-core/partmap/gpt.c:117: GPT entry 8: start=8370176, length=16384
grub-probe: info: no LDM signature found.
grub-probe: info: Scanning for lvm devices on disk hostdisk//dev/sdb.
grub-probe: info: no LVM signature found.
grub-core/partmap/gpt.c:93: Read a valid GPT header
grub-core/partmap/gpt.c:117: GPT entry 0: start=2048, length=8368128
grub-core/disk/diskfilter.c:137: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb
grub-probe: info: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid1x devices on disk hostdisk//dev/sdb.
grub-core/osdep/hostdisk.c:416: opening the device `/dev/sdb1' in open_device()
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb1'
grub-probe: info: Scanning for mdraid09 devices on disk hostdisk//dev/sdb.
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb1'
grub-probe: info: Scanning for mdraid09_be devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for dmraid_nv devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for ldm devices on disk hostdisk//dev/sdb.
grub-probe: info: scanning hostdisk//dev/sdb for LDM.
grub-probe: info: no LDM signature found.
grub-probe: info: Scanning for lvm devices on disk hostdisk//dev/sdb.
grub-probe: info: no LVM signature found.
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:117: GPT entry 8: start=8370176, length=16384
grub-core/disk/diskfilter.c:137: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb
grub-probe: info: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid1x devices on disk hostdisk//dev/sdb.
grub-core/osdep/hostdisk.c:416: opening the device `/dev/sdb9' in open_device()
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb9'
grub-probe: info: Scanning for mdraid09 devices on disk hostdisk//dev/sdb.
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb9'
grub-probe: info: Scanning for mdraid09_be devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for dmraid_nv devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for ldm devices on disk hostdisk//dev/sdb.
grub-probe: info: scanning hostdisk//dev/sdb for LDM.
grub-probe: info: no LDM signature found.
grub-probe: info: Scanning for lvm devices on disk hostdisk//dev/sdb.
grub-probe: info: no LVM signature found.
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdb'.
grub-core/partmap/gpt.c:93: Read a valid GPT header
grub-core/partmap/gpt.c:117: GPT entry 0: start=2048, length=8368128
grub-probe: info: Partition 0 starts from 2048.
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdb'.
grub-probe: info: /dev/sdb1 is present.
grub-probe: info: Looking for /dev/sdb1.
grub-probe: info: /dev/sdb is a parent of /dev/sdb1.
grub-probe: info: /dev/sdb1 starts from 2048.
grub-probe: info: opening the device hostdisk//dev/sdb.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdb'...
grub-probe: info: drive = 0.
grub-probe: info: the size of hostdisk//dev/sdb is 8388608.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdb'...
grub-probe: info: drive = 0.
grub-probe: info: the size of hostdisk//dev/sdb is 8388608.
grub-core/disk/diskfilter.c:137: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb
grub-probe: info: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid1x devices on disk hostdisk//dev/sdb.
grub-core/osdep/hostdisk.c:416: opening the device `/dev/sdb' in open_device()
grub-probe: info: Scanning for mdraid09 devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid09_be devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for dmraid_nv devices on disk hostdisk//dev/sdb.
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb'
grub-probe: info: Scanning for ldm devices on disk hostdisk//dev/sdb.
grub-probe: info: scanning hostdisk//dev/sdb for LDM.
grub-core/partmap/gpt.c:93: Read a valid GPT header
grub-core/partmap/gpt.c:117: GPT entry 0: start=2048, length=8368128
grub-core/partmap/gpt.c:117: GPT entry 8: start=8370176, length=16384
grub-probe: info: no LDM signature found.
grub-probe: info: Scanning for lvm devices on disk hostdisk//dev/sdb.
grub-probe: info: no LVM signature found.
grub-core/partmap/gpt.c:93: Read a valid GPT header
grub-core/partmap/gpt.c:117: GPT entry 0: start=2048, length=8368128
grub-core/disk/diskfilter.c:137: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb
grub-probe: info: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid1x devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid09 devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid09_be devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for dmraid_nv devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for ldm devices on disk hostdisk//dev/sdb.
grub-probe: info: scanning hostdisk//dev/sdb for LDM.
grub-probe: info: no LDM signature found.
grub-probe: info: Scanning for lvm devices on disk hostdisk//dev/sdb.
grub-probe: info: no LVM signature found.
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:117: GPT entry 8: start=8370176, length=16384
grub-core/disk/diskfilter.c:137: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb
grub-probe: info: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid1x devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid09 devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid09_be devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for dmraid_nv devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for ldm devices on disk hostdisk//dev/sdb.
grub-probe: info: scanning hostdisk//dev/sdb for LDM.
grub-probe: info: no LDM signature found.
grub-probe: info: Scanning for lvm devices on disk hostdisk//dev/sdb.
grub-probe: info: no LVM signature found.
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdb'.
grub-core/partmap/gpt.c:93: Read a valid GPT header
grub-core/partmap/gpt.c:117: GPT entry 0: start=2048, length=8368128
grub-probe: info: Partition 0 starts from 2048.
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdb'.
grub-probe: info: /dev/sdb1 is present.
grub-probe: info: Looking for /dev/sdb1.
grub-probe: info: /dev/sdb is a parent of /dev/sdb1.
grub-probe: info: /dev/sdb1 starts from 2048.
grub-probe: info: opening the device hostdisk//dev/sdb.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdb'...
grub-probe: info: drive = 0.
grub-probe: info: the size of hostdisk//dev/sdb is 8388608.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdb'...
grub-probe: info: drive = 0.
grub-probe: info: the size of hostdisk//dev/sdb is 8388608.
grub-core/disk/diskfilter.c:137: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb
grub-probe: info: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid1x devices on disk hostdisk//dev/sdb.
grub-core/osdep/hostdisk.c:416: opening the device `/dev/sdb' in open_device()
grub-probe: info: Scanning for mdraid09 devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid09_be devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for dmraid_nv devices on disk hostdisk//dev/sdb.
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb'
grub-probe: info: Scanning for ldm devices on disk hostdisk//dev/sdb.
grub-probe: info: scanning hostdisk//dev/sdb for LDM.
grub-core/partmap/gpt.c:93: Read a valid GPT header
grub-core/partmap/gpt.c:117: GPT entry 0: start=2048, length=8368128
grub-core/partmap/gpt.c:117: GPT entry 8: start=8370176, length=16384
grub-probe: info: no LDM signature found.
grub-probe: info: Scanning for lvm devices on disk hostdisk//dev/sdb.
grub-probe: info: no LVM signature found.
grub-core/partmap/gpt.c:93: Read a valid GPT header
grub-core/partmap/gpt.c:117: GPT entry 0: start=2048, length=8368128
grub-core/disk/diskfilter.c:137: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb
grub-probe: info: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid1x devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid09 devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid09_be devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for dmraid_nv devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for ldm devices on disk hostdisk//dev/sdb.
grub-probe: info: scanning hostdisk//dev/sdb for LDM.
grub-probe: info: no LDM signature found.
grub-probe: info: Scanning for lvm devices on disk hostdisk//dev/sdb.
grub-probe: info: no LVM signature found.
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:117: GPT entry 8: start=8370176, length=16384
grub-core/disk/diskfilter.c:137: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb
grub-probe: info: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid1x devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid09 devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid09_be devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for dmraid_nv devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for ldm devices on disk hostdisk//dev/sdb.
grub-probe: info: scanning hostdisk//dev/sdb for LDM.
grub-probe: info: no LDM signature found.
grub-probe: info: Scanning for lvm devices on disk hostdisk//dev/sdb.
grub-probe: info: no LVM signature found.
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdb'.
grub-core/partmap/gpt.c:93: Read a valid GPT header
grub-core/partmap/gpt.c:117: GPT entry 0: start=2048, length=8368128
grub-probe: info: Partition 0 starts from 2048.
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdb'.
grub-probe: info: opening hostdisk//dev/sdb,gpt1.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdb,gpt1'...
grub-probe: info: drive = 0.
grub-probe: info: the size of hostdisk//dev/sdb is 8388608.
grub-core/partmap/gpt.c:93: Read a valid GPT header
grub-core/partmap/gpt.c:117: GPT entry 0: start=2048, length=8368128
grub-core/kern/fs.c:56: Detecting zfs...
grub-core/fs/zfs/zfs.c:4697: vdev_disk_read_rootlabel: label[0] 16384
grub-core/osdep/hostdisk.c:416: opening the device `/dev/sdb1' in open_device()
grub-core/fs/zfs/zfs.c:689: check_feature: com.delphix:hole_birth ok
grub-core/fs/zfs/zfs.c:689: check_feature: com.delphix:embedded_data ok
grub-core/fs/zfs/zfs.c:5208: zfs_mount: attempting to mount "rpool" from hostdisk//dev/sdb,gpt1
grub-core/fs/zfs/zfs.c:5267: pool guid: f15a21cad080bd9c
grub-core/fs/zfs/zfs.c:717: scan_devices: proc
grub-core/kern/disk.c:196: Opening `proc'...
grub-core/fs/zfs/zfs.c:4697: vdev_disk_read_rootlabel: label[0] 16384
grub-core/kern/disk.c:421: Read out of range: sector 0x20 (attempt to read or write outside of disk `proc').
grub-core/fs/zfs/zfs.c:4697: vdev_disk_read_rootlabel: label[1] 278528
grub-core/kern/disk.c:421: Read out of range: sector 0x220 (attempt to read or write outside of disk `proc').
grub-core/fs/zfs/zfs.c:4697: vdev_disk_read_rootlabel: label[2] 18446744073709043712
grub-core/kern/disk.c:421: Read out of range: sector 0x7ffffffffffc20 (attempt to read or write outside of disk `proc').
grub-core/fs/zfs/zfs.c:4697: vdev_disk_read_rootlabel: label[3] 18446744073709305856
grub-core/kern/disk.c:421: Read out of range: sector 0x7ffffffffffe20 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:295: Closing `proc'.
grub-core/kern/disk.c:196: Opening `proc'...
grub-core/kern/disk.c:421: Read out of range: sector 0x1 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x1 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x1 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x1 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x6 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:295: Closing `proc'.
grub-core/fs/zfs/zfs.c:717: scan_devices: hostdisk//dev/sdb
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdb'...
grub-probe: info: drive = 0.
grub-probe: info: the size of hostdisk//dev/sdb is 8388608.
grub-core/fs/zfs/zfs.c:4697: vdev_disk_read_rootlabel: label[0] 16384
grub-core/osdep/hostdisk.c:416: opening the device `/dev/sdb' in open_device()
grub-core/fs/zfs/zfs.c:4697: vdev_disk_read_rootlabel: label[1] 278528
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb'
grub-core/fs/zfs/zfs.c:4697: vdev_disk_read_rootlabel: label[2] 4294459392
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb'
grub-core/fs/zfs/zfs.c:4697: vdev_disk_read_rootlabel: label[3] 4294721536
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb'
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdb'.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdb'...
grub-probe: info: drive = 0.
grub-probe: info: the size of hostdisk//dev/sdb is 8388608.
grub-core/partmap/gpt.c:93: Read a valid GPT header
grub-core/partmap/gpt.c:117: GPT entry 0: start=2048, length=8368128
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:117: GPT entry 8: start=8370176, length=16384
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdb'.
grub-core/fs/zfs/zfs.c:717: scan_devices: hostdisk//dev/sdb,gpt9
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdb,gpt9'...
grub-probe: info: drive = 0.
grub-probe: info: the size of hostdisk//dev/sdb is 8388608.
grub-core/partmap/gpt.c:93: Read a valid GPT header
grub-core/partmap/gpt.c:117: GPT entry 0: start=2048, length=8368128
grub-core/partmap/gpt.c:117: GPT entry 8: start=8370176, length=16384
grub-core/fs/zfs/zfs.c:4697: vdev_disk_read_rootlabel: label[0] 16384
grub-core/osdep/hostdisk.c:416: opening the device `/dev/sdb9' in open_device()
grub-core/fs/zfs/zfs.c:4697: vdev_disk_read_rootlabel: label[1] 278528
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb9'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb9'
grub-core/fs/zfs/zfs.c:4697: vdev_disk_read_rootlabel: label[2] 7880704
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb9'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb9'
grub-core/fs/zfs/zfs.c:4697: vdev_disk_read_rootlabel: label[3] 8142848
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb9'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb9'
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdb'.
grub-core/fs/zfs/zfs.c:717: scan_devices: hostdisk//dev/sdb,gpt1
grub-core/fs/zfs/zfs.c:4697: vdev_disk_read_rootlabel: label[0] 16384
grub-core/fs/zfs/zfs.c:803: disk from our pool: hostdisk//dev/sdb,gpt1
grub-core/fs/zfs/zfs.c:4872: vdev id: 0 guid: 749cc39de9cf70
grub-core/fs/zfs/zfs.c:4891: adding vdev: 0 guid: 749cc39de9cf70
grub-core/fs/zfs/zfs.c:4831: vdev_add: ashift: 12
grub-core/fs/zfs/zfs.c:4968: adding child: 0 guid: 30495cc8cdbbceae
grub-core/fs/zfs/zfs.c:4815: vdev_add: added disk hostdisk//dev/sdb,gpt1, state: 0 original: 1
grub-core/fs/zfs/zfs.c:4831: vdev_add: ashift: 9
grub-core/kern/disk.c:196: Opening `proc'...
grub-core/disk/diskfilter.c:137: Scanning for DISKFILTER devices on disk proc
grub-probe: info: Scanning for DISKFILTER devices on disk proc.
grub-probe: info: Scanning for mdraid1x devices on disk proc.
grub-core/kern/disk.c:421: Read out of range: sector 0xfffffffffffffff0 (attempt to read or write outside of disk `proc').
grub-probe: info: Scanning for mdraid09 devices on disk proc.
grub-core/kern/disk.c:421: Read out of range: sector 0xffffffffffffff80 (attempt to read or write outside of disk `proc').
grub-probe: info: Scanning for mdraid09_be devices on disk proc.
grub-core/kern/disk.c:421: Read out of range: sector 0xffffffffffffff80 (attempt to read or write outside of disk `proc').
grub-probe: info: Scanning for dmraid_nv devices on disk proc.
grub-core/kern/disk.c:421: Read out of range: sector 0xfffffffffffffffe (attempt to read or write outside of disk `proc').
grub-probe: info: Scanning for ldm devices on disk proc.
grub-probe: info: scanning proc for LDM.
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-probe: info: no LDM signature found.
grub-probe: info: Scanning for lvm devices on disk proc.
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x1 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x1 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x1 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x1 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x0 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:421: Read out of range: sector 0x6 (attempt to read or write outside of disk `proc').
grub-core/kern/disk.c:295: Closing `proc'.
grub-core/kern/disk.c:196: Opening `hostdisk//dev/sdb'...
grub-probe: info: drive = 0.
grub-probe: info: the size of hostdisk//dev/sdb is 8388608.
grub-core/disk/diskfilter.c:137: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb
grub-probe: info: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid1x devices on disk hostdisk//dev/sdb.
grub-core/osdep/hostdisk.c:416: opening the device `/dev/sdb' in open_device()
grub-probe: info: Scanning for mdraid09 devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid09_be devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for dmraid_nv devices on disk hostdisk//dev/sdb.
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb'
grub-probe: info: Scanning for ldm devices on disk hostdisk//dev/sdb.
grub-probe: info: scanning hostdisk//dev/sdb for LDM.
grub-core/partmap/gpt.c:93: Read a valid GPT header
grub-core/partmap/gpt.c:117: GPT entry 0: start=2048, length=8368128
grub-core/partmap/gpt.c:117: GPT entry 8: start=8370176, length=16384
grub-probe: info: no LDM signature found.
grub-probe: info: Scanning for lvm devices on disk hostdisk//dev/sdb.
grub-probe: info: no LVM signature found.
grub-core/partmap/gpt.c:93: Read a valid GPT header
grub-core/partmap/gpt.c:117: GPT entry 0: start=2048, length=8368128
grub-core/disk/diskfilter.c:137: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb
grub-probe: info: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid1x devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid09 devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid09_be devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for dmraid_nv devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for ldm devices on disk hostdisk//dev/sdb.
grub-probe: info: scanning hostdisk//dev/sdb for LDM.
grub-probe: info: no LDM signature found.
grub-probe: info: Scanning for lvm devices on disk hostdisk//dev/sdb.
grub-probe: info: no LVM signature found.
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/gpt.c:117: GPT entry 8: start=8370176, length=16384
grub-core/disk/diskfilter.c:137: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb
grub-probe: info: Scanning for DISKFILTER devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid1x devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid09 devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for mdraid09_be devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for dmraid_nv devices on disk hostdisk//dev/sdb.
grub-probe: info: Scanning for ldm devices on disk hostdisk//dev/sdb.
grub-probe: info: scanning hostdisk//dev/sdb for LDM.
grub-probe: info: no LDM signature found.
grub-probe: info: Scanning for lvm devices on disk hostdisk//dev/sdb.
grub-probe: info: no LVM signature found.
grub-core/partmap/dfly.c:72: bad magic (found 0x0; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/partmap/dfly.c:72: bad magic (found 0x20494645; wanted 0xc4464c59)
grub-core/partmap/apple.c:123: bad magic (found 0x0; wanted 0x4552)
grub-core/kern/disk.c:295: Closing `hostdisk//dev/sdb'.
grub-core/fs/zfs/zfs.c:5374: zfs_mount failed: missing devices
grub-core/fs/zfs/zfs.c:4554: zfs_unmount: 0x233ab90
grub-core/kern/fs.c:78: zfs detection failed.
grub-core/kern/fs.c:56: Detecting xfs...
grub-core/kern/fs.c:78: xfs detection failed.
grub-core/kern/fs.c:56: Detecting ufs2...
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb1'
grub-core/kern/fs.c:78: ufs2 detection failed.
grub-core/kern/fs.c:56: Detecting ufs1_be...
grub-core/kern/fs.c:78: ufs1_be detection failed.
grub-core/kern/fs.c:56: Detecting ufs1...
grub-core/kern/fs.c:78: ufs1 detection failed.
grub-core/kern/fs.c:56: Detecting udf...
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb1'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb1'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb1'
grub-core/osdep/hostdisk.c:395: reusing open device `/dev/sdb1'
grub-core/kern/fs.c:78: udf detection failed.
grub-core/kern/fs.c:56: Detecting tarfs...
grub-core/kern/fs.c:78: tarfs detection failed.
grub-core/kern/fs.c:56: Detecting squash4...
grub-core/kern/fs.c:78: squash4 detection failed.
grub-core/kern/fs.c:56: Detecting sfs...
grub-core/kern/fs.c:78: sfs detection failed.
grub-core/kern/fs.c:56: Detecting romfs...
grub-core/kern/fs.c:78: romfs detection failed.
grub-core/kern/fs.c:56: Detecting reiserfs...
grub-core/kern/fs.c:78: reiserfs detection failed.
grub-core/kern/fs.c:56: Detecting procfs...
grub-core/kern/fs.c:78: procfs detection failed.
grub-core/kern/fs.c:56: Detecting odc...
grub-core/kern/fs.c:78: odc detection failed.
grub-core/kern/fs.c:56: Detecting ntfs...
grub-core/kern/fs.c:78: ntfs detection failed.
grub-core/kern/fs.c:56: Detecting nilfs2...
grub-core/kern/fs.c:78: nilfs2 detection failed.
grub-core/kern/fs.c:56: Detecting newc...
grub-core/kern/fs.c:78: newc detection failed.
grub-core/kern/fs.c:56: Detecting minix_be...
grub-core/kern/fs.c:78: minix_be detection failed.
grub-core/kern/fs.c:56: Detecting minix3_be...
grub-core/kern/fs.c:78: minix3_be detection failed.
grub-core/kern/fs.c:56: Detecting minix3...
grub-core/kern/fs.c:78: minix3 detection failed.
grub-core/kern/fs.c:56: Detecting minix2_be...
grub-core/kern/fs.c:78: minix2_be detection failed.
grub-core/kern/fs.c:56: Detecting minix2...
grub-core/kern/fs.c:78: minix2 detection failed.
grub-core/kern/fs.c:56: Detecting minix...
grub-core/kern/fs.c:78: minix detection failed.
grub-core/kern/fs.c:56: Detecting jfs...
grub-core/kern/fs.c:78: jfs detection failed.
grub-core/kern/fs.c:56: Detecting iso9660...
grub-core/kern/fs.c:78: iso9660 detection failed.
grub-core/kern/fs.c:56: Detecting hfsplus...
grub-core/kern/fs.c:78: hfsplus detection failed.
grub-core/kern/fs.c:56: Detecting hfs...
grub-core/kern/fs.c:78: hfs detection failed.
grub-core/kern/fs.c:56: Detecting fat...
grub-core/kern/fs.c:78: fat detection failed.
grub-core/kern/fs.c:56: Detecting ext2...
grub-core/kern/fs.c:78: ext2 detection failed.
grub-core/kern/fs.c:56: Detecting exfat...
grub-core/kern/fs.c:78: exfat detection failed.
grub-core/kern/fs.c:56: Detecting cpiofs_be...
grub-core/kern/fs.c:78: cpiofs_be detection failed.
grub-core/kern/fs.c:56: Detecting cpiofs...
grub-core/kern/fs.c:78: cpiofs detection failed.
grub-core/kern/fs.c:56: Detecting cbfs...
grub-core/kern/disk.c:421: Read out of range: sector 0x8bb3de (attempt to read or write outside of partition).
grub-core/kern/fs.c:78: cbfs detection failed.
grub-core/kern/fs.c:56: Detecting btrfs...
grub-core/kern/fs.c:78: btrfs detection failed.
grub-core/kern/fs.c:56: Detecting bfs...
grub-core/kern/fs.c:78: bfs detection failed.
grub-core/kern/fs.c:56: Detecting afs...
grub-core/kern/fs.c:78: afs detection failed.
grub-core/kern/fs.c:56: Detecting affs...
grub-core/kern/fs.c:78: affs detection failed.
grub-probe: error: unknown filesystem.
tsoome commented 7 years ago

this log is telling "grub-core/fs/zfs/zfs.c:5374: zfs_mount failed: missing devices", you need to present on command line at least minimum amount of devices depending on pool layout, for mirror its 1 disk, for 2+1 raidz, its 2 disks etc. for hint, either grub-probe or grub-testfs did need the -c X to tell it how many disk devices are you providing (again sorry, its some time since I did use grub2;)

Spongman commented 7 years ago

I'm just using the command that is executed by /etc/grub.d/10_linux during update-grub. if grub-probe needs all the devices, then shouldn't 10_linux be passing them to it?

tsoome commented 7 years ago

yes it should; also I only now noticed you have posted the pool layout - you have raidz from 5 disks, so you need to provide at least 4 (better all 5), grub-probe is basically user space file system reader, but it will need devices specified on command line, it wont pick them by itself. Thats easy to test anyhow.

Spongman commented 7 years ago

indeed. grub-probe returns zfs if i pass any 4 of the 5 devices.

also, grub-probe --target=device correctly returns the following:

$ grub-probe --target=device /
/dev/sdb1
/dev/sdd1
/dev/sde1
/dev/sdc1
/dev/sda1

which means the following line in /usr/sbin/grub-mkconfig:

GRUB_DEVICE="`${grub_probe} --target=device / | head -n1`"

...along with all the logic that assumes a single device... is obviously broken.

tsoome commented 7 years ago

yap, well, at least thats simple to fix and would be good to notify debian devs as well:)

Spongman commented 7 years ago

looks like they've known about this for over 7 years now... https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=540786

morph027 commented 7 years ago

Ran into the same issue with current xenial:

All info from inside chrooted installer

root@ubuntu:/# dpkg -l | grep grub
ii  grub-common                        2.02~beta2-36ubuntu3.2          amd64        GRand Unified Bootloader (common files)
ii  grub-gfxpayload-lists              0.7                             amd64        GRUB gfxpayload blacklist
ii  grub-pc                            2.02~beta2-36ubuntu3.2          amd64        GRand Unified Bootloader, version 2 (PC/BIOS version)
ii  grub-pc-bin                        2.02~beta2-36ubuntu3.2          amd64        GRand Unified Bootloader, version 2 (PC/BIOS binaries)
ii  grub2                              2.02~beta2-36ubuntu3.2          amd64        GRand Unified Bootloader, version 2 (dummy package)
ii  grub2-common                       2.02~beta2-36ubuntu3.2          amd64        GRand Unified Bootloader (common files for version 2)
root@ubuntu:/# zfs list
NAME                USED  AVAIL  REFER  MOUNTPOINT
rpool               842M  29.9G    96K  /mnt
rpool/ROOT          669M  29.9G    96K  none
rpool/ROOT/ubuntu   669M  29.9G   669M  /mnt
rpool/home          216K  29.9G    96K  /mnt/home
rpool/home/root     120K  29.9G   120K  /mnt/root
rpool/var           172M  29.9G    96K  /mnt/var
rpool/var/cache     171M  29.9G   171M  /mnt/var/cache
rpool/var/log       220K  29.9G   220K  /mnt/var/log
rpool/var/spool      96K  29.9G    96K  /mnt/var/spool
rpool/var/tmp       104K  29.9G   104K  /mnt/var/tmp
root@ubuntu:/# mount | grep zfs
rpool/ROOT/ubuntu on / type zfs (rw,noatime,xattr,noacl)
rpool/home on /home type zfs (rw,nosuid,noatime,xattr,noacl)
rpool/home/root on /root type zfs (rw,nosuid,noatime,xattr,noacl)
rpool/var/cache on /var/cache type zfs (rw,nosuid,noexec,noatime,xattr,noacl)
rpool/var/log on /var/log type zfs (rw,nosuid,noexec,noatime,xattr,noacl)
rpool/var/spool on /var/spool type zfs (rw,nosuid,noexec,noatime,xattr,noacl)
rpool/var/tmp on /var/tmp type zfs (rw,nosuid,noatime,xattr,noacl)
root@ubuntu:/# grub-probe -v /
grub-probe: info: cannot open `/boot/grub/device.map': No such file or directory.
grub-probe: info: /dev/vda is not present.
grub-probe: info: Looking for /dev/vda.
grub-probe: info: /dev/vda is a parent of /dev/vda.
grub-probe: info: /dev/vda is present.
grub-probe: info: Looking for /dev/vda.
grub-probe: info: /dev/vda is a parent of /dev/vda.
grub-probe: info: /dev/vda is present.
grub-probe: info: Looking for /dev/vda.
grub-probe: info: /dev/vda is a parent of /dev/vda.
grub-probe: info: opening hostdisk//dev/vda.
grub-probe: info: drive = 0.
grub-probe: info: the size of hostdisk//dev/vda is 67108864.
grub-probe: error: unknown filesystem.
tsoome commented 7 years ago

Yea, and the diagnostics is all the same use --target=device and verbose/debug options if this is not enough. And if this is the same issue, bug your vendor till they fix it...

morph027 commented 7 years ago

Ok, not bothering this one

...for the record: https://github.com/zfsonlinux/zfs/issues/5448

muggenhor commented 7 years ago

I'm having the same (or similar) problem. Where for a striped pool (no mirrors, no raidz) with as only enabled feature lz4 compression grub-probe -d /dev/dm-0 succeeds but grub-probe -d /dev/dm-1 fails with unknown filesystem.

Unfortunately for me this is more problematic than just the config files for grub not getting auto-generated properly (I can edit those if need be). It seems I've got similar problems when trying to unlock the LUKS disks before GRUB starts. Specifically the cryptomount -u $UUID command succeeds at unlocking one of the disks, but complains about "unknown compression type" or something like that for the second, dropping in the rescue prompt immediately afterwards. For some reason this used to work out of the box with a single-disk setup but not anymore...

Edit: this is the exact output I get:

Attempting to decrypt master key...
Enter passphrase for hd0,gpt1 ($UUID1):
Slot 0 opened
Attempting to decrypt master key...
Enter passphrase for hd1,gpt1 ($UUID2):
error: access denied.
error: no such cryptodisk found.
error: unknown filesystem.
Entering rescue mode...
grub rescue> _

When trying again:

Attempting to decrypt master key...
Enter passphrase for hd0,gpt1 ($UUID1):
Slot 0 opened
Attempting to decrypt master key...
Enter passphrase for hd1,gpt1 ($UUID2):
error: unknown filesystem.
Entering rescue mode...
grub rescue> _

Both were run when this was the load.cfg:

cryptomount -u $UUID1
cryptomount -u $UUID2

End edit

As a result my current boot procedure consists of using a live USB stick, installing cryptsetup and zfs-dkms to the live install, unlocking all my disks, importing my pool (/boot is on there) and using kexec to load the kernel and initrd image. Any suggestions to try out are welcome.

This is the relevant part of grub.cfg (for some reason at runtime it executes the cryptomount -u calls in reverse btw):

insmod luks
insmod zfs

cryptomount -u <disk-0-uuid>
cryptomount -u <disk-1-uuid>
set root='cryptouuid/<disk-0-uuid>
cryptouuid/<disk-1-uuid>'

For grub-probe -vvv the significant difference in output seems to be that for both disks I see this line:

grub-core/fs/zfs/zfs.c:2496: looking for 'features_for_read'

But only for the disk that works do I ever see this line following it:

grub-core/fs/zfs/zfs.c:2496: looking for 'root_dataset'

Additionally for that disk I see it detecting the lz4 feature, which I don't for the disk failing detection:

grub-core/fs/zfs/zfs.c:2112: zap: name = org.illumos:lz4_compress, value = 1, cd = 0
tsoome commented 7 years ago

The unknown compression (or checksum failure) hints that the zfs reader is reading wrong blocks for some reason - so it did read something and trying to make sense what it got - but the compression type bits do not match the list of supported compression algorithm.

muggenhor commented 7 years ago

@tsoome maybe the problem is that I'm using a multi-disk setup and the pool as a whole shouldn't be expected to be readable until both disks are decrypted? Because it always gives this error after entering the passphrase to unlock the first disk but never after the second. Unfortunately after the second it just drops into the rescue prompt.

Hmm, maybe the prefix is the problem? Because that looks like this:

prefix=(cryptouuid/$UUID1)/ROOT/Debian-Jessie@/boot/grub

I'm not seeing the second disk in there. Should it be? Or if it's decrypted should it be smart enough to figure out to use one of those on its own?

muggenhor commented 7 years ago

Some more experiments from within GRUB at the bootprompt give me this (I have to record a video and manually enter it so I'm restricting myself to what I believe is most useful):

Apparently the cryptomount commands work. But trying to access the pool of 2 disks afterwards fails:

grub> zfsinfo crypto0
error: Unsupported features in pool.

Running the same command with debug=all:

grub> set debug=all
grub> zfsinfo crypto0
...
fs/zfs/zfs.c:1192: label ok 0
fs/zfs/zfs.c:1007: check 2 passed
fs/zfs/zfs.c:1018: check 3 passed
fs/zfs/zfs.c:1025: check 4 passed
fs/zfs/zfs.c:1035: check 6 passed
fs/zfs/zfs.c:1043: check 7 passed
fs/zfs/zfs.c:1054: check 8 passed
fs/zfs/zfs.c:1064: check 9 passed
fs/zfs/zfs.c:1086: check 11 passed
fs/zfs/zfs.c:1112: check 10 passed
fs/zfs/zfs.c:1137: check 12 passed (feature flags)
...
fs/zfs/zfs.c:2496: looking for 'features_for_read'
...
kern/disk.c:196: Opening `crypto1'...
disk/cryptodisk.c:532: Opening device crypto1
fs/zfs/zfs.c:1192: label ok 0
fs/zfs/zfs.c:1007: check 2 passed
fs/zfs/zfs.c:1018: check 3 passed
fs/zfs/zfs.c:1025: check 4 passed
fs/zfs/zfs.c:1035: check 6 passed
fs/zfs/zfs.c:1043: check 7 passed
fs/zfs/zfs.c:1054: check 8 passed
fs/zfs/zfs.c:1064: check 9 passed
fs/zfs/zfs.c:1086: check 11 passed
fs/zfs/zfs.c:1112: check 10 passed
fs/zfs/zfs.c:1137: check 12 passed (feature flags)
...
fs/zfs/zfs.c:2506: zap read
fs/zfs/zfs.c:2519: fat zap
fs/zfs/zfs.c:2023: endian = 1
fs/zfs/zfs.c:2049: endian = 1
fs/zfs/zfs.c:1878: zio_read: E 0: size 16384/4096
fs/zfs/zfs.c:1899: endian = 1
fs/zfs/zfs.c:595: dva=100000008, 16c4cc3d8
fs/zfs/zfs.c:442: checksum fletcher4 verification failed
fs/zfs/zfs.c:447: actual checksum xxx yyy zzz www
fs/zfs/zfs.c:452: expected checksum uuu rrr sss ttt
fs/zfs/zfs.c:1922: incorrect checksum
fs/zfs/zfs.c:2523: returned 9
kern/disk.c:295: Closing `crypto`'.
disk/cryptodisk.c:551: Closing disk
...
fs/zfs/zfs.c:3393: incorrect nvlist header
Pool name: unavailable
fs/zfs/zfs.c:3393: incorrect nvlist header
Pool GUID: unavailable
fs/zfs/zfs.c:3393: incorrect nvlist header
Unable to retrieve pool state
fs/zfs/zfs.c:3393: incorrect nvlist header
No virtual device tree available

I find it strange that with more debug output I never find the original Unsupported features in pool message in the output. Secondly, the checksum mismatch, could that be the cause of the incorrect nvlist header message afterwards?

The checksum message is still strange because A: I've got sha256 set as the checksum property of the root dataset, but maybe for some pool-wide things it falls back to fletcher4 anyway, in which case it's still weird that a scrub is perfectly capable of passing while reporting 0 errors.

Any ideas how to debug this more? Get more info that can help with a solution?

PS: This didn't happen before I added my second disk (i.e. before I had a single-disk pool).

andrey-ivanov commented 5 years ago

Just faced an another "unknown filesystem" case. GRUB detection does not support ashift=14. Simplest test case is 

zpool create -o ashift=14 -R /mnt/data data /dev/nvme0n1
zfs create -o mountpoint=/data data/data
grub-probe /mnt/data/

With ashift 13 grub-probe works.

$ grub-probe --version
grub-probe (GRUB) 2.02-2ubuntu8.9
Shellcat-Zero commented 2 years ago

I just had this issue on Focal after following the guide. Not sure how to debug this.