search

zfsonlinux / grub

GRUB enhancements for ZFS on Linux
16 stars 18 forks source link

grub-probe: error: unknown filesystem. #28

Closed Popyllol closed 4 years ago

Popyllol commented 5 years ago

Hi, first of all, I saw the other issues belonging to this error message and hardcoding rpool="rpool" in /etc/grub.d/10_linux is a valid workaround, but no solution.

Back to the problem: The

  rpool=`${grub_probe} --device ${GRUB_DEVICE} --target=fs_label 2>/dev/null || true`

leads to an empty rpool variable which ends up in an

 LINUX_ROOT_DEVICE="ZFS=${rpool}${bootfs}"
                        ^^^^^^^^ empty
 LINUX_ROOT_DEVICE="ZFS=${bootfs}"

which is unbootable.

My environment is Ubuntu 18.04.3 with a threeway bootpool and rootpool:

# grub-probe --version
grub-probe (GRUB) 2.02-2ubuntu8.13
# zpool status bpool rpool
  pool: bpool
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
    still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
    the pool may no longer be accessible by software that does not support
    the features. See zpool-features(5) for details.
  scan: resilvered 102M in 0h0m with 0 errors on Wed Aug 14 13:23:15 2019
config:

    NAME                              STATE     READ WRITE CKSUM
    bpool                             ONLINE       0     0     0
      mirror-0                        ONLINE       0     0     0
        scsi-3500003965811f688-part3  ONLINE       0     0     0
        scsi-35000039658121890-part3  ONLINE       0     0     0
        scsi-3500003965811f6c0-part3  ONLINE       0     0     0

errors: No known data errors

  pool: rpool
 state: ONLINE
  scan: resilvered 875M in 0h1m with 0 errors on Wed Aug 14 13:25:00 2019
config:

    NAME                              STATE     READ WRITE CKSUM
    rpool                             ONLINE       0     0     0
      mirror-0                        ONLINE       0     0     0
        scsi-3500003965811f688-part4  ONLINE       0     0     0
        scsi-35000039658121890-part4  ONLINE       0     0     0
        scsi-3500003965811f6c0-part4  ONLINE       0     0     0

errors: No known data errors
# zpool status -LP bpool rpool
  pool: bpool
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
    still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
    the pool may no longer be accessible by software that does not support
    the features. See zpool-features(5) for details.
  scan: resilvered 102M in 0h0m with 0 errors on Wed Aug 14 13:23:15 2019
config:

    NAME           STATE     READ WRITE CKSUM
    bpool          ONLINE       0     0     0
      mirror-0     ONLINE       0     0     0
        /dev/sda3  ONLINE       0     0     0
        /dev/sdc3  ONLINE       0     0     0
        /dev/sdb3  ONLINE       0     0     0

errors: No known data errors

  pool: rpool
 state: ONLINE
  scan: resilvered 875M in 0h1m with 0 errors on Wed Aug 14 13:25:00 2019
config:

    NAME           STATE     READ WRITE CKSUM
    rpool          ONLINE       0     0     0
      mirror-0     ONLINE       0     0     0
        /dev/sda4  ONLINE       0     0     0
        /dev/sdc4  ONLINE       0     0     0
        /dev/sdb4  ONLINE       0     0     0

errors: No known data errors
# zpool get ashift rpool bpool
NAME   PROPERTY  VALUE   SOURCE
bpool  ashift    12      local
rpool  ashift    12      local

Debug output of grub-probe (yes, it is called with all three devices in /etc/grub.d/10_linux):

# /usr/sbin/grub-probe --device /dev/sda4 /dev/sdc4 /dev/sdb4 --target=fs_label -vvv 2>&1 | grep zfs
grub-core/kern/fs.c:56: Detecting zfs...
grub-core/fs/zfs/zfs.c:1192: label ok 0
grub-core/fs/zfs/zfs.c:1007: check 2 passed
grub-core/fs/zfs/zfs.c:1018: check 3 passed
grub-core/fs/zfs/zfs.c:1025: check 4 passed
grub-core/fs/zfs/zfs.c:1035: check 6 passed
grub-core/fs/zfs/zfs.c:1043: check 7 passed
grub-core/fs/zfs/zfs.c:1054: check 8 passed
grub-core/fs/zfs/zfs.c:1064: check 9 passed
grub-core/fs/zfs/zfs.c:1086: check 11 passed
grub-core/fs/zfs/zfs.c:1112: check 10 passed
grub-core/fs/zfs/zfs.c:1128: str=com.delphix:hole_birth
grub-core/fs/zfs/zfs.c:1128: str=com.delphix:embedded_data
grub-core/fs/zfs/zfs.c:1137: check 12 passed (feature flags)
grub-core/fs/zfs/zfs.c:1878: zio_read: E 0: size 2048/2048
grub-core/fs/zfs/zfs.c:1899: endian = -1
grub-core/fs/zfs/zfs.c:595: dva=8, 4c78b88
grub-core/fs/zfs/zfs.c:2681: endian = -1, blkid=0
grub-core/fs/zfs/zfs.c:2023: endian = -1
grub-core/fs/zfs/zfs.c:2054: endian = -1
grub-core/fs/zfs/zfs.c:1878: zio_read: E 0: size 131072/4096
grub-core/fs/zfs/zfs.c:1899: endian = -1
grub-core/fs/zfs/zfs.c:595: dva=8, 4c78b80
grub-core/fs/zfs/zfs.c:2023: endian = 1
grub-core/fs/zfs/zfs.c:2049: endian = 1
grub-core/fs/zfs/zfs.c:1878: zio_read: E 0: size 16384/4096
grub-core/fs/zfs/zfs.c:1899: endian = 1
grub-core/fs/zfs/zfs.c:595: dva=8, 4c76fd8
grub-core/fs/zfs/zfs.c:2685: alive
grub-core/fs/zfs/zfs.c:2496: looking for 'features_for_read'
grub-core/fs/zfs/zfs.c:2023: endian = 1
grub-core/fs/zfs/zfs.c:2049: endian = 1
grub-core/fs/zfs/zfs.c:1878: zio_read: E 0: size 16384/4096
grub-core/fs/zfs/zfs.c:1899: endian = 1
grub-core/fs/zfs/zfs.c:595: dva=8, 380
grub-core/fs/zfs/zfs.c:2506: zap read
grub-core/fs/zfs/zfs.c:2519: fat zap
grub-core/fs/zfs/zfs.c:2023: endian = 1
grub-core/fs/zfs/zfs.c:2049: endian = 1
grub-core/fs/zfs/zfs.c:1878: zio_read: E 0: size 16384/4096
grub-core/fs/zfs/zfs.c:1899: endian = 1
grub-core/fs/zfs/zfs.c:595: dva=8, 1ce0fd8
grub-core/fs/zfs/zfs.c:2279: fzap: length 18
grub-core/fs/zfs/zfs.c:2523: returned 0
grub-core/fs/zfs/zfs.c:2681: endian = -1, blkid=1
grub-core/fs/zfs/zfs.c:2023: endian = -1
grub-core/fs/zfs/zfs.c:2054: endian = -1
grub-core/fs/zfs/zfs.c:1878: zio_read: E 0: size 131072/4096
grub-core/fs/zfs/zfs.c:1899: endian = -1
grub-core/fs/zfs/zfs.c:595: dva=8, 4c78b80
grub-core/fs/zfs/zfs.c:2023: endian = 1
grub-core/fs/zfs/zfs.c:2049: endian = 1
grub-core/fs/zfs/zfs.c:1878: zio_read: E 0: size 16384/4096
grub-core/fs/zfs/zfs.c:1899: endian = 1
grub-core/fs/zfs/zfs.c:595: dva=8, 4c78b78
grub-core/fs/zfs/zfs.c:2685: alive
grub-core/fs/zfs/zfs.c:2023: endian = 1
grub-core/fs/zfs/zfs.c:2049: endian = 1
grub-core/fs/zfs/zfs.c:1878: zio_read: E 0: size 1024/1024
grub-core/fs/zfs/zfs.c:1899: endian = 1
grub-core/fs/zfs/zfs.c:595: dva=8, 14c0
grub-core/fs/zfs/zfs.c:2112: zap: name = org.illumos:lz4_compress, value = 1, cd = 0
grub-core/fs/zfs/zfs.c:2112: zap: name = com.joyent:multi_vdev_crash_dump, value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2112: zap: name = com.delphix:hole_birth, value = 1, cd = 0
grub-core/fs/zfs/zfs.c:2112: zap: name = com.delphix:extensible_dataset, value = c, cd = 0
grub-core/fs/zfs/zfs.c:2112: zap: name = com.delphix:embedded_data, value = 1, cd = 0
grub-core/fs/zfs/zfs.c:2112: zap: name = org.open-zfs:large_blocks, value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2112: zap: name = org.zfsonlinux:large_dnode, value = c, cd = 0
grub-core/kern/fs.c:78: zfs detection failed.
# zhack feature stat rpool
for_read_obj:
    org.illumos:edonr = 0
    com.delphix:extensible_dataset = 12
    org.illumos:lz4_compress = 1
    org.open-zfs:large_blocks = 0
    com.delphix:embedded_data = 1
    org.illumos:skein = 0
    com.joyent:multi_vdev_crash_dump = 0
    org.zfsonlinux:large_dnode = 12
    com.delphix:hole_birth = 1
    org.illumos:sha512 = 0
for_write_obj:
    com.delphix:spacemap_histogram = 62
    com.delphix:bookmarks = 0
    com.delphix:async_destroy = 0
    org.zfsonlinux:userobj_accounting = 12
    com.delphix:empty_bpobj = 12
    com.joyent:filesystem_limits = 0
    com.delphix:enabled_txg = 12
descriptions_obj:
    com.joyent:filesystem_limits = Filesystem and snapshot limits.
    org.illumos:sha512 = SHA-512/256 hash algorithm.
    org.zfsonlinux:userobj_accounting = User/Group object accounting.
    com.delphix:embedded_data = Blocks which compress very well use even less space.
    org.open-zfs:large_blocks = Support for blocks larger than 128KB.
    org.illumos:skein = Skein hash algorithm.
    com.delphix:async_destroy = Destroy filesystems asynchronously.
    com.delphix:empty_bpobj = Snapshots use less space.
    com.delphix:extensible_dataset = Enhanced dataset functionality, used by other features.
    org.illumos:lz4_compress = LZ4 compression algorithm support.
    org.zfsonlinux:large_dnode = Variable on-disk size of dnodes.
    com.joyent:multi_vdev_crash_dump = Crash dumps to multiple vdev pools.
    com.delphix:spacemap_histogram = Spacemaps maintain space histograms.
    com.delphix:bookmarks = "zfs bookmark" command
    com.delphix:enabled_txg = Record txg at which a feature is enabled
    org.illumos:edonr = Edon-R hash algorithm.
    com.delphix:hole_birth = Retain hole birth txg for more precise zfs send
enabled_txg_obj:
    org.zfsonlinux:userobj_accounting = 4
    org.zfsonlinux:large_dnode = 4
    org.illumos:skein = 4
    org.illumos:edonr = 4
    com.delphix:extensible_dataset = 4
    com.delphix:bookmarks = 4
    org.illumos:sha512 = 4
    com.delphix:hole_birth = 4
    com.delphix:enabled_txg = 4
    com.joyent:filesystem_limits = 4
    com.delphix:embedded_data = 4
    org.open-zfs:large_blocks = 4
label config:
    com.delphix:hole_birth
    com.delphix:embedded_data
# zhack feature stat bpool
for_read_obj:
    com.delphix:hole_birth = 1
    com.delphix:extensible_dataset = 2
    org.illumos:lz4_compress = 1
    org.open-zfs:large_blocks = 0
    com.delphix:embedded_data = 1
for_write_obj:
    org.zfsonlinux:userobj_accounting = 2
    com.joyent:filesystem_limits = 0
    com.delphix:async_destroy = 0
    com.delphix:spacemap_histogram = 34
    com.delphix:enabled_txg = 7
    com.delphix:empty_bpobj = 2
    com.delphix:bookmarks = 0
descriptions_obj:
    com.delphix:embedded_data = Blocks which compress very well use even less space.
    org.open-zfs:large_blocks = Support for blocks larger than 128KB.
    com.delphix:spacemap_histogram = Spacemaps maintain space histograms.
    com.delphix:bookmarks = "zfs bookmark" command
    com.delphix:enabled_txg = Record txg at which a feature is enabled
    com.delphix:hole_birth = Retain hole birth txg for more precise zfs send
    com.delphix:empty_bpobj = Snapshots use less space.
    com.delphix:extensible_dataset = Enhanced dataset functionality, used by other features.
    org.zfsonlinux:userobj_accounting = User/Group object accounting.
    org.illumos:lz4_compress = LZ4 compression algorithm support.
    com.delphix:async_destroy = Destroy filesystems asynchronously.
    com.joyent:filesystem_limits = Filesystem and snapshot limits.
enabled_txg_obj:
    com.delphix:hole_birth = 4
    org.zfsonlinux:userobj_accounting = 4
    com.delphix:spacemap_histogram = 4
    org.open-zfs:large_blocks = 4
    org.illumos:lz4_compress = 4
    com.joyent:filesystem_limits = 4
    com.delphix:enabled_txg = 4
label config:
    com.delphix:embedded_data
    com.delphix:hole_birth