openzfs / zfs

OpenZFS on Linux and FreeBSD
https://openzfs.github.io/openzfs-docs
Other
10.61k stars 1.75k forks source link

Still having Issue 3708 #13391

Open altimmons opened 2 years ago

altimmons commented 2 years ago

System information

MEMORY160 GiB DDR3 Multi-bit ECC Utilization: 11% |   Intel® Xeon® CPU E5-2687W 0 @ 3.10GHz x2 ASUSTeK Computer INC. Z9PR-D12 Series, Version Rev 1.xxAmerican Megatrends Inc., Version 6002BIOS dated: Wed 18 Apr 2018 12:00:00 AM EDT

KTN-STL3 x 3 (15 drive shelf) SAS-3 Serial Attached SCSI controller: Broadcom / LSI SAS2116 PCI-Express Fusion-MPT SAS-2 [Meteor] (rev 02) 60 or so drives, NVME, SAS, and SATA3, SATA2

--

Type | Version/Name Distribution Name | Slackware Distribution Version | Version: 6.9.2 Kernel Version | 5.10.28-Unraid Architecture | x64 OpenZFS Version | zfs-2.1.4-1 zfs-kmod-2.1.4-1

Describe the problem you're observing

Duplicate of #3708 Now Closed, but stubbornly persisting.

Disappointing to see this 2016 bug still around. Adding by ID fails, and Ive tried 1000 ways over 4 days, deleting MBRs, and Partitions and whatever else I can think of.

What worked was adding by /dev/sdx (which we are explicitly advised NOT to do) and then using dracwyrm 7 year work around.

Really in 7 years no one has fixed this?

specifically I get this error

missing link: sdbs was partitioned but /dev/disk/by-vdev/ztest8-part1 is missing

I set up aliases in /etc/zfs/vdev_id.conf and running udevadm trigger then: ❯ zpool create -f ztst mirror ztest8 ztest9

But also using the /dev/disk/by-id/ names fails every time, though I think the error differs slightly:

cannot label 'sdbt': failed to detect device partitions on '/dev/sdbt1': 19

Describe how to reproduce the problem

❯ zpool create -f ztst mirror ztest8 ztest9

Include any warning/errors/backtraces from the system logs

Apr 29 18:25:05 Unraid zed[25659]: zfs_deliver_dle: device '16208818218211078488' not found
Apr 29 18:25:05 Unraid zed[25659]: zfs_slm_event: EC_dev_status.dev_dle
Apr 29 18:25:05 Unraid zed[25659]: #011vdev_guid: 16208818218211078488
Apr 29 18:25:05 Unraid zed[25659]: #011pool_guid: 7065604327842627958
Apr 29 18:25:05 Unraid zed[25659]: #011dev_size: 1000194703360
Apr 29 18:25:05 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:25:05 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:25:05 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:25:05 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:25:05 Unraid zed[25659]: #011subclass: dev_dle
Apr 29 18:25:05 Unraid zed[25659]: #011class: EC_dev_status
Apr 29 18:25:05 Unraid zed[25659]: zed_disk_event:
Apr 29 18:25:05 Unraid zed[25659]: zed_udev_monitor: /dev/sdbs9 sectors 16384 < 131072 (minimum)
Apr 29 18:25:04 Unraid zed[25659]: zed_udev_monitor: skip /dev/sdbs since it has a gpt partition already
Apr 29 18:25:04 Unraid zed[25659]: zed_udev_monitor: /dev/sdbs9 sectors 16384 < 131072 (minimum)
Apr 29 18:25:04 Unraid zed[25659]: zfs_deliver_add: adding ata-HUA722010CLA330_J82TVSVV-part1 (pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0) (is_slice 1)
Apr 29 18:25:04 Unraid zed[25659]: zfs_slm_event: EC_dev_add.disk
Apr 29 18:25:04 Unraid zed[25659]: #011vdev_guid: 16208818218211078488
Apr 29 18:25:04 Unraid zed[25659]: #011pool_guid: 7065604327842627958
Apr 29 18:25:04 Unraid zed[25659]: #011dev_size: 1000194703360
Apr 29 18:25:04 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:25:04 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:25:04 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:25:04 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:25:04 Unraid zed[25659]: #011subclass: disk
Apr 29 18:25:04 Unraid zed[25659]: #011class: EC_dev_add
Apr 29 18:25:04 Unraid zed[25659]: zed_disk_event:
Apr 29 18:25:04 Unraid unassigned.devices: Disk with ID 'HUA722010CLA330_J82TVSVV (sdbs)' is not set to auto mount.
Apr 29 18:25:04 Unraid zed[25659]: zed_udev_monitor: skip /dev/sdbs since it has a gpt partition already
Apr 29 18:25:04 Unraid zed[25659]: zed_udev_monitor: skip /dev/sdbs since it has a gpt partition already
Apr 29 18:25:03 Unraid zed[25659]: zed_udev_monitor: /dev/sdbs9 sectors 16384 < 131072 (minimum)
Apr 29 18:25:03 Unraid zed[25659]: Retire Agent: zfs_retire_recv: 'resource.fs.zfs.removed'
Apr 29 18:25:03 Unraid zed[25659]: Diagnosis Engine: discarding 'resource.fs.zfs.removed for vdev 16208818218211078488
Apr 29 18:25:03 Unraid zed[25659]: agent post event: mapping 'EC_dev_remove' to 'resource.fs.zfs.removed'
Apr 29 18:25:03 Unraid zed[25659]: #011vdev_guid: 16208818218211078488
Apr 29 18:25:03 Unraid zed[25659]: #011pool_guid: 7065604327842627958
Apr 29 18:25:03 Unraid zed[25659]: #011dev_size: 1000194703360
Apr 29 18:25:03 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:25:03 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:25:03 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:25:03 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:25:03 Unraid zed[25659]: #011subclass: disk
Apr 29 18:25:03 Unraid zed[25659]: #011class: EC_dev_remove
Apr 29 18:25:03 Unraid zed[25659]: zed_disk_event:
Apr 29 18:25:03 Unraid zed[25659]: zed_udev_monitor: /dev/sdbs9 sectors 16384 < 131072 (minimum)
Apr 29 18:25:03 Unraid zed[25659]: zfs_deliver_add: adding ata-HUA722010CLA330_J82TVSVV-part1 (pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0) (is_slice 1)
Apr 29 18:25:03 Unraid zed[25659]: zfs_slm_event: EC_dev_add.disk
Apr 29 18:25:03 Unraid zed[25659]: #011vdev_guid: 16208818218211078488
Apr 29 18:25:03 Unraid zed[25659]: #011pool_guid: 7065604327842627958
Apr 29 18:25:03 Unraid zed[25659]: #011dev_size: 1000194703360
Apr 29 18:25:03 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:25:03 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:25:03 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:25:03 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:25:03 Unraid zed[25659]: #011subclass: disk
Apr 29 18:25:03 Unraid zed[25659]: #011class: EC_dev_add
Apr 29 18:25:03 Unraid zed[25659]: zed_disk_event:
Apr 29 18:25:03 Unraid unassigned.devices: Disk with ID 'HUA722010CLA330_J82TVSVV (sdbs)' is not set to auto mount.
Apr 29 18:25:03 Unraid zed[25659]: zed_udev_monitor: skip /dev/sdbs since it has a gpt partition already
Apr 29 18:25:03 Unraid kernel: sdbs: sdbs1 sdbs9
Apr 29 18:25:03 Unraid zed[25659]: Retire Agent: zfs_retire_recv: 'resource.fs.zfs.removed'
Apr 29 18:25:03 Unraid zed[25659]: Diagnosis Engine: discarding 'resource.fs.zfs.removed for vdev 0
Apr 29 18:25:03 Unraid zed[25659]: agent post event: mapping 'EC_dev_remove' to 'resource.fs.zfs.removed'
Apr 29 18:25:03 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:25:03 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:25:03 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:25:03 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:25:03 Unraid zed[25659]: #011subclass: disk
Apr 29 18:25:03 Unraid zed[25659]: #011class: EC_dev_remove
Apr 29 18:25:03 Unraid zed[25659]: zed_disk_event:
Apr 29 18:25:03 Unraid kernel: sdbs: sdbs1 sdbs9
Apr 29 18:25:03 Unraid kernel: Alternate GPT is invalid, using primary GPT.
Apr 29 18:25:00 Unraid zed[25659]: zfs_deliver_add: adding ata-HUA722010CLA330_J82TVSVV-part1 (pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0) (is_slice 1)
Apr 29 18:25:00 Unraid zed[25659]: zfs_slm_event: EC_dev_add.disk
Apr 29 18:25:00 Unraid zed[25659]: #011dev_size: 1000203820544
Apr 29 18:25:00 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:25:00 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:25:00 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:25:00 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:25:00 Unraid zed[25659]: #011subclass: disk
Apr 29 18:25:00 Unraid zed[25659]: #011class: EC_dev_add
Apr 29 18:25:00 Unraid zed[25659]: zed_disk_event:
Apr 29 18:25:00 Unraid zed[25659]: zed_udev_monitor: skip /dev/sdbs since it has a gpt partition already
Apr 29 18:25:00 Unraid zed[25659]: Retire Agent: zfs_retire_recv: 'resource.fs.zfs.removed'
Apr 29 18:25:00 Unraid zed[25659]: Diagnosis Engine: discarding 'resource.fs.zfs.removed for vdev 0
Apr 29 18:25:00 Unraid zed[25659]: agent post event: mapping 'EC_dev_remove' to 'resource.fs.zfs.removed'
Apr 29 18:25:00 Unraid zed[25659]: #011dev_size: 1000203820544
Apr 29 18:25:00 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:25:00 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:25:00 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:25:00 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:25:00 Unraid zed[25659]: #011subclass: disk
Apr 29 18:25:00 Unraid zed[25659]: #011class: EC_dev_remove
Apr 29 18:25:00 Unraid zed[25659]: zed_disk_event:
Apr 29 18:25:00 Unraid zed[25659]: zfs_deliver_add: adding ata-HUA722010CLA330_J82TVSVV-part1 (pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0) (is_slice 1)
Apr 29 18:25:00 Unraid zed[25659]: zfs_slm_event: EC_dev_add.disk
Apr 29 18:25:00 Unraid zed[25659]: #011dev_size: 1000203820544
Apr 29 18:25:00 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:25:00 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:25:00 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:25:00 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:25:00 Unraid zed[25659]: #011subclass: disk
Apr 29 18:25:00 Unraid zed[25659]: #011class: EC_dev_add
Apr 29 18:25:00 Unraid zed[25659]: zed_disk_event:
Apr 29 18:25:00 Unraid zed[25659]: zed_udev_monitor: skip /dev/sdbs since it has a gpt partition already
Apr 29 18:25:00 Unraid zed[25659]: zed_udev_monitor: /dev/sdbs9 sectors 16384 < 131072 (minimum)
Apr 29 18:25:00 Unraid zed[25659]: Retire Agent: zfs_retire_recv: 'resource.fs.zfs.removed'
Apr 29 18:25:00 Unraid zed[25659]: Diagnosis Engine: discarding 'resource.fs.zfs.removed for vdev 16208818218211078488
Apr 29 18:25:00 Unraid zed[25659]: agent post event: mapping 'EC_dev_remove' to 'resource.fs.zfs.removed'
Apr 29 18:25:00 Unraid zed[25659]: #011vdev_guid: 16208818218211078488
Apr 29 18:25:00 Unraid zed[25659]: #011pool_guid: 7065604327842627958
Apr 29 18:25:00 Unraid zed[25659]: #011dev_size: 1000203820544
Apr 29 18:25:00 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:25:00 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:25:00 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:25:00 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:25:00 Unraid zed[25659]: #011subclass: disk
Apr 29 18:25:00 Unraid zed[25659]: #011class: EC_dev_remove
Apr 29 18:25:00 Unraid zed[25659]: zed_disk_event:
Apr 29 18:25:00 Unraid kernel: sdbs: sdbs1
Apr 29 18:25:00 Unraid kernel: sdbs: sdbs1
Apr 29 18:24:31 Unraid zed[25659]: zfs_deliver_add: adding ata-HUA722010CLA330_J82TVSVV-part1 (pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0) (is_slice 1)
Apr 29 18:24:31 Unraid zed[25659]: zfs_slm_event: EC_dev_add.disk
Apr 29 18:24:31 Unraid zed[25659]: #011vdev_guid: 16208818218211078488
Apr 29 18:24:31 Unraid zed[25659]: #011pool_guid: 7065604327842627958
Apr 29 18:24:31 Unraid zed[25659]: #011dev_size: 1000194703360
Apr 29 18:24:31 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:24:31 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:24:31 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:24:31 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:24:31 Unraid zed[25659]: #011subclass: disk
Apr 29 18:24:31 Unraid zed[25659]: #011class: EC_dev_add
Apr 29 18:24:31 Unraid zed[25659]: zed_disk_event:
Apr 29 18:24:31 Unraid unassigned.devices: Disk with ID 'HUA722010CLA330_J82TVSVV (sdbs)' is not set to auto mount.
Apr 29 18:24:31 Unraid zed[25659]: zed_udev_monitor: /dev/sdbs9 sectors 16384 < 131072 (minimum)
Apr 29 18:24:31 Unraid zed[25659]: zed_udev_monitor: skip /dev/sdbs since it has a gpt partition already
Apr 29 18:24:31 Unraid zed[25659]: zed_udev_monitor: /dev/sdbs9 sectors 16384 < 131072 (minimum)
Apr 29 18:24:31 Unraid zed[25659]: Retire Agent: zfs_retire_recv: 'resource.fs.zfs.removed'
Apr 29 18:24:31 Unraid zed[25659]: Diagnosis Engine: discarding 'resource.fs.zfs.removed for vdev 16208818218211078488
Apr 29 18:24:31 Unraid zed[25659]: agent post event: mapping 'EC_dev_remove' to 'resource.fs.zfs.removed'
Apr 29 18:24:31 Unraid zed[25659]: #011vdev_guid: 16208818218211078488
Apr 29 18:24:31 Unraid zed[25659]: #011pool_guid: 7065604327842627958
Apr 29 18:24:31 Unraid zed[25659]: #011dev_size: 1000194703360
Apr 29 18:24:31 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:24:31 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:24:31 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:24:31 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:24:31 Unraid zed[25659]: #011subclass: disk
Apr 29 18:24:31 Unraid zed[25659]: #011class: EC_dev_remove
Apr 29 18:24:31 Unraid zed[25659]: zed_disk_event:
Apr 29 18:24:31 Unraid kernel: sdbs: sdbs1 sdbs9
Apr 29 18:23:52 Unraid zed[25659]: zed_udev_monitor: /dev/sdbs9 sectors 16384 < 131072 (minimum)
 29 18:23:52 Unraid zed[25659]: #011vdev_guid: 16208818218211078488
Apr 29 18:23:52 Unraid zed[25659]: #011pool_guid: 7065604327842627958
Apr 29 18:23:52 Unraid zed[25659]: #011dev_size: 1000194703360
Apr 29 18:23:52 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:23:52 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:23:52 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:23:52 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:23:52 Unraid zed[25659]: #011subclass: disk
Apr 29 18:23:52 Unraid zed[25659]: #011class: EC_dev_add
Apr 29 18:23:52 Unraid zed[25659]: zed_disk_event:
Apr 29 18:23:52 Unraid unassigned.devices: Disk with ID 'HUA722010CLA330_J82TVSVV (sdbs)' is not set to auto mount.
Apr 29 18:23:52 Unraid zed[25659]: zed_udev_monitor: skip /dev/sdbs since it has a gpt partition already
Apr 29 18:23:52 Unraid zed[25659]: zed_udev_monitor: /dev/sdbs9 sectors 16384 < 131072 (minimum)
Apr 29 18:23:52 Unraid zed[25659]: Retire Agent: zfs_retire_recv: 'resource.fs.zfs.removed'
Apr 29 18:23:52 Unraid zed[25659]: Diagnosis Engine: discarding 'resource.fs.zfs.removed for vdev 16208818218211078488
Apr 29 18:23:52 Unraid zed[25659]: agent post event: mapping 'EC_dev_remove' to 'resource.fs.zfs.removed'
Apr 29 18:23:52 Unraid zed[25659]: #011vdev_guid: 16208818218211078488
Apr 29 18:23:52 Unraid zed[25659]: #011pool_guid: 7065604327842627958
Apr 29 18:23:52 Unraid zed[25659]: #011dev_size: 1000194703360
Apr 29 18:23:52 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:23:52 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:23:52 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:23:52 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:23:52 Unraid zed[25659]: #011subclass: disk
Apr 29 18:23:52 Unraid zed[25659]: #011class: EC_dev_remove
Apr 29 18:23:52 Unraid zed[25659]: zed_disk_event:
Apr 29 18:23:52 Unraid zed[25659]: zed_udev_monitor: /dev/sdbs9 sectors 16384 < 131072 (minimum)
Apr 29 18:23:52 Unraid zed[25659]: zfs_deliver_add: adding ata-HUA722010CLA330_J82TVSVV-part1 (pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0) (is_slice 1)
Apr 29 18:23:52 Unraid zed[25659]: zfs_slm_event: EC_dev_add.disk
Apr 29 18:23:52 Unraid zed[25659]: #011vdev_guid: 16208818218211078488
Apr 29 18:23:52 Unraid zed[25659]: #011pool_guid: 7065604327842627958
Apr 29 18:23:52 Unraid zed[25659]: #011dev_size: 1000194703360
Apr 29 18:23:52 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:23:52 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:23:52 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:23:52 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:23:52 Unraid zed[25659]: #011subclass: disk
Apr 29 18:23:52 Unraid zed[25659]: #011class: EC_dev_add
\Apr 29 18:23:52 Unraid zed[25659]: zed_disk_event:
Apr 29 18:23:52 Unraid unassigned.devices: Disk with ID 'HUA722010CLA330_J82TVSVV (sdbs)' is not set to auto mount.
Apr 29 18:23:51 Unraid zed[25659]: zed_udev_monitor: skip /dev/sdbs since it has a gpt partition already
Apr 29 18:23:51 Unraid zed[25659]: zed_udev_monitor: /dev/sdbs9 sectors 16384 < 131072 (minimum)
Apr 29 18:23:51 Unraid zed[25659]: zed_udev_monitor: /dev/sdbs9 sectors 16384 < 131072 (minimum)
Apr 29 18:23:51 Unraid zed[25659]: Retire Agent: zfs_retire_recv: 'resource.fs.zfs.removed'
Apr 29 18:23:51 Unraid zed[25659]: Diagnosis Engine: discarding 'resource.fs.zfs.removed for vdev 16208818218211078488
Apr 29 18:23:51 Unraid zed[25659]: agent post event: mapping 'EC_dev_remove' to 'resource.fs.zfs.removed'
Apr 29 18:23:51 Unraid zed[25659]: #011vdev_guid: 16208818218211078488
Apr 29 18:23:51 Unraid zed[25659]: #011pool_guid: 7065604327842627958
Apr 29 18:23:51 Unraid zed[25659]: #011dev_size: 1000194703360
Apr 29 18:23:51 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:23:51 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:23:51 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:23:51 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:23:51 Unraid zed[25659]: #011subclass: disk
Apr 29 18:23:51 Unraid zed[25659]: #011class: EC_dev_remove
Apr 29 18:23:51 Unraid zed[25659]: zed_disk_event:
Apr 29 18:23:51 Unraid zed[25659]: zfs_deliver_add: adding ata-HUA722010CLA330_J82TVSVV-part1 (pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0) (is_slice 1)
Apr 29 18:23:51 Unraid zed[25659]: zfs_slm_event: EC_dev_add.disk
Apr 29 18:23:51 Unraid zed[25659]: #011vdev_guid: 16208818218211078488
Apr 29 18:23:51 Unraid zed[25659]: #011pool_guid: 7065604327842627958
Apr 29 18:23:51 Unraid zed[25659]: #011dev_size: 1000194703360
Apr 29 18:23:51 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:23:51 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:23:51 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:23:51 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:23:51 Unraid zed[25659]: #011subclass: disk
Apr 29 18:23:51 Unraid zed[25659]: #011class: EC_dev_add
Apr 29 18:23:51 Unraid zed[25659]: zed_disk_event:
Apr 29 18:23:51 Unraid unassigned.devices: Disk with ID 'HUA722010CLA330_J82TVSVV (sdbs)' is not set to auto mount.
Apr 29 18:23:51 Unraid zed[25659]: zed_udev_monitor: skip /dev/sdbs since it has a gpt partition already
Apr 29 18:23:51 Unraid zed[25659]: zed_udev_monitor: /dev/sdbs9 sectors 16384 < 131072 (minimum)
Apr 29 18:23:51 Unraid zed[25659]: Retire Agent: zfs_retire_recv: 'resource.fs.zfs.removed'
Apr 29 18:23:51 Unraid zed[25659]: Diagnosis Engine: discarding 'resource.fs.zfs.removed for vdev 16208818218211078488
Apr 29 18:23:51 Unraid zed[25659]: agent post event: mapping 'EC_dev_remove' to 'resource.fs.zfs.removed'
Apr 29 18:23:51 Unraid zed[25659]: #011vdev_guid: 16208818218211078488
Apr 29 18:23:51 Unraid zed[25659]: #011pool_guid: 7065604327842627958
Apr 29 18:23:51 Unraid zed[25659]: #011dev_size: 1000194703360
Apr 29 18:23:51 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:23:51 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:23:51 Unraid zed[25659]: #011vdev_guid: 16208818218211078488
Apr 29 18:23:51 Unraid zed[25659]: #011pool_guid: 7065604327842627958
Apr 29 18:23:51 Unraid zed[25659]: #011dev_size: 1000194703360
Apr 29 18:23:51 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:23:51 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:23:51 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target12:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:23:51 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:23:51 Unraid zed[25659]: #011subclass: disk
Apr 29 18:23:51 Unraid zed[25659]: #011class: EC_dev_add
Apr 29 18:23:51 Unraid zed[25659]: zed_disk_event:
Apr 29 18:23:51 Unraid unassigned.devices: Disk with ID 'HUA722010CLA330_J82TVSVV (sdbs)' is not set to auto mount.
Apr 29 18:23:51 Unraid zed[25659]: zed_udev_monitor: skip /dev/sdbs since it has a gpt partition already
Apr 29 18:23:51 Unraid zed[25659]: zed_udev_monitor: /dev/sdbs9 sectors 16384 < 131072 (minimum)
Apr 29 18:23:51 Unraid zed[25659]: Retire Agent: zfs_retire_recv: 'resource.fs.zfs.removed'
Apr 29 18:23:51 Unraid zed[25659]: Diagnosis Engine: discarding 'resource.fs.zfs.removed for vdev 16208818218211078488
Apr 29 18:23:51 Unraid zed[25659]: agent post event: mapping 'EC_dev_remove' to 'resource.fs.zfs.removed'
Apr 29 18:23:51 Unraid zed[25659]: #011vdev_guid: 16208818218211078488
Apr 29 18:23:51 Unraid zed[25659]: #011pool_guid: 7065604327842627958
Apr 29 18:23:51 Unraid zed[25659]: #011dev_size: 1000194703360
Apr 29 18:23:51 Unraid zed[25659]: #011phys_path: pci-0000:85:00.0-sas-exp0x5006048003e336bf-phy15-lun-0
Apr 29 18:23:51 Unraid zed[25659]: #011devid: ata-HUA722010CLA330_J82TVSVV-part1
Apr 29 18:23:51 Unraid zed[25659]: #011path: /devices/pci0000:80/0000:80:03.0/0000:85:00.0/host12/port-12:0/expander-12:0/port-12:0:24/end_device-12:0:24/target1
2:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:23:51 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:23:51 Unraid zed[25659]: #011subclass: disk
Apr 29 18:23:51 Unraid zed[25659]: #011class: EC_dev_remove
Apr 29 18:23:51 Unraid zed[25659]: zed_disk_event:
Apr 29 18:23:51 Unraid kernel: sdbs: sdbs1 sdbs9
Apr 29 18:23:51 Unraid kernel: sdbs: sdbs1 sdbs9
Apr 29 18:23:51 Unraid kernel: sdbs: sdbs1 sdbs9
Apr 29 18:23:51 Unraid kernel: Alternate GPT is invalid, using primary GPT.
Apr 29 18:23:49 Unraid emhttpd: no mountpoint along path: /mnt/user/system/docker
Apr 29 18:23:49 Unraid winbindd[2655]:   daemon_ready: daemon 'winbindd' finished starting up and ready to serve connections
Apr 29 18:23:49 Unraid winbindd[2655]: [2022/04/29 18:23:49.611573,  0] ../../lib/util/become_daemon.c:135(daemon_ready)
Apr 29 18:23:49 Unraid emhttpd: shcmd (2795262): cp /tmp/emhttp/smb.service /etc/avahi/services/smb.service
Apr 29 18:23:49 Unraid winbindd[2655]:   initialize_winbindd_cache: clearing cache and re-creating with version number 2
Apr 29 18:23:49 Unraid winbindd[2655]: [2022/04/29 18:23:49.610933,  0] ../../source3/winbindd/winbindd_cache.c:3203(initialize_winbindd_cache)
Apr 29 18:23:49 Unraid root:                  /usr/sbin/winbindd -D
Apr 29 18:23:49 Unraid nmbd[2633]:   daemon_ready: daemon 'nmbd' finished starting up and ready to serve connections
Apr 29 18:23:49 Unraid nmbd[2633]: [2022/04/29 18:23:49.562718,  0] ../../lib/util/become_daemon.c:135(daemon_ready)
Apr 29 18:23:49 Unraid root:                  /usr/sbin/wsdd 
Apr 29 18:23:49 Unraid smbd[2619]:   daemon_ready: daemon 'smbd' finished starting up and ready to serve connections
Apr 29 18:23:49 Unraid smbd[2619]: [2022/04/29 18:23:49.548179,  0] ../../lib/util/become_daemon.c:135(daemon_ready)
Apr 29 18:23:49 Unraid root:                  /usr/sbin/nmbd -D
Apr 29 18:23:49 Unraid root: Starting Samba:  /usr/sbin/smbd -D
Apr 29 18:23:49 Unraid emhttpd: shcmd (2795261): /etc/rc.d/rc.samba start
Apr 29 18:23:49 Unraid emhttpd: Starting services...
Apr 29 18:23:49 Unraid emhttpd: shcmd (2795259): sync
Apr 29 18:23:49 Unraid emhttpd: shcmd (2795258): rm -f /boot/config/forcesync
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sdaj
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sda
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sday
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sdp
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sdam
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sdi
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sdbb
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sdl
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sdbf
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sdbi
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sdax
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sdo
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/nvme1n1
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sdan
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sdbm
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sdba
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sdbl
Apr 29 18:23:48 Unraid emhttpd: read SMART /dev/sdaq
2:0:61/12:0:61:0/block/sdbs/sdbs1
Apr 29 18:23:51 Unraid zed[25659]: #011dev_name: /dev/sdbs1
Apr 29 18:23:51 Unraid zed[25659]: #011subclass: disk
Apr 29 18:23:51 Unraid zed[25659]: #011class: EC_dev_remove
Apr 29 18:23:51 Unraid zed[25659]: zed_disk_event:
Apr 29 18:23:51 Unraid kernel: sdbs: sdbs1 sdbs9
Apr 29 18:23:51 Unraid kernel: sdbs: sdbs1 sdbs9
Apr 29 18:23:51 Unraid kernel: sdbs: sdbs1 sdbs9
Apr 29 18:23:51 Unraid kernel: Alternate GPT is invalid, using primary 
rincebrain commented 2 years ago

This does work for many people, so while I'm sorry it doesn't work for you, and don't immediately know why, it's not, as far as I can tell, the case that it doesn't work in general, or I think we'd have rather a lot more gnashing of teeth.

Could you possibly provide more specifics about how you setup vdev_id.conf, and/or what specific disk IDs in by-id you were using that don't work but work if you use /dev/sdX? Is that log from you using /dev/sdbs directly, or referencing it by-id or the like?

The last two times I saw something fail like this were when Ubuntu installed a handler in the udev "hey a disk appeared" rules that triggered a Python program every time a disk event happened, so that various things that expected an infinitely short delay between "I modified /dev/sdx" and "/dev/sdx1 appeared" broke, and when I was playing games with manually symlinking device names to lie about a disk whose path in by-id had changed, and did not create the matching -partX links.

nabijaczleweli commented 2 years ago

Notably, we do actually wait for udev for DISK_LABEL_WAIT (30s), and, indeed, that error is only printed when that wait fails; assuming you've configured with libudev, zpool_label_disk_wait() returns ENODEV iff the device resolves, udev_device_is_ready(), and one of the stat64()s on its udev_device_get_devlinks_list_entry()ies fails (without libudev it's the same but it Just tries to stat the path for 30 seconds). Can you post a strace -f over the failing command?

altimmons commented 2 years ago

I will shortly, I have another zpool to set up probably tomorrow. Adding it by /dev/sdX then converting it to by id seems easier than writing out a vdev.conf file anyway ANy

On Sat, Apr 30, 2022 at 11:12 AM наб @.***> wrote:

Notably, we do actually wait for udev for DISK_LABEL_WAIT (30s), and, indeed, that error is only printed when that wait fails; assuming you've configured with libudev, zpool_label_disk_wait() returns ENODEV iff the device resolves, udev_device_is_ready(), and one of the stat64()s on its udev_device_get_devlinks_list_entry()ies fails (without udev it's the same but it Just tries to stat the path for 30 seconds). Can you post a strace -f over the failing command?

— Reply to this email directly, view it on GitHub https://github.com/openzfs/zfs/issues/13391#issuecomment-1114004372, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADU23JIIX3NUPYU7TLMYQLTVHVEVTANCNFSM5UXWKVGQ . You are receiving this because you authored the thread.Message ID: @.***>

stale[bot] commented 1 year ago

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.