[FEATURE] Support for iscsi boot disks

[ibrokethecloud]

Is your feature request related to a problem? Please describe.

Large customers running diskless systems may wish to use something like an iscsi disk as the boot device.

Most modern hardware allows definition of an iscsi target as a boot device.

As we have tested internally when booting of an iscsi disk additional kernel arguments rd.iscsi.firmware rd.iscsi.ibft are needed to ensure that the OS can boot successfully.

Describe the solution you'd like

We should have an additional check in the installer / upgrade path to detect if the install disk is an iscsi disk, and accordingly append these additional kernel arguments in the default entry.

These arguments will need to be retained post upgrade as well.

Describe alternatives you've considered

Additional context

[abonillabeeche]

We may need to include the support to use the OS and DATA mpio disks rather than the individual disk. This means the initramfs would need mpio support. See here

Otherwise, we'd need to disable mpio for the root/OS disk.

[innobead]

Let's add HEP for this, even though we have a KB for reference.

cc @bk201

[harvesterhci-io-github-bot]

Pre Ready-For-Testing Checklist

• [ ] If labeled: require/HEP Has the Harvester Enhancement Proposal PR submitted? The HEP PR is at:

• [ ] Where is the reproduce steps/test steps documented? The reproduce steps/test steps are at:

• [ ] Is there a workaround for the issue? If so, where is it documented? The workaround is at:

• [ ] Have the backend code been merged (harvester, harvester-installer, etc) (including backport-needed/*)? The PR is at:

  • [ ] Does the PR include the explanation for the fix or the feature?

  • [ ] Does the PR include deployment change (YAML/Chart)? If so, where are the PRs for both YAML file and Chart? The PR for the YAML change is at: The PR for the chart change is at:

• [ ] If labeled: area/ui Has the UI issue filed or ready to be merged? The UI issue/PR is at:

• [ ] If labeled: require/doc, require/knowledge-base Has the necessary document PR submitted or merged? The documentation/KB PR is at:

• [ ] If NOT labeled: not-require/test-plan Has the e2e test plan been merged? Have QAs agreed on the automation test case? If only test case skeleton w/o implementation, have you created an implementation issue?

  • The automation skeleton PR is at:
  • The automation test case PR is at:

• [ ] If the fix introduces the code for backward compatibility Has a separate issue been filed with the label release/obsolete-compatibility? The compatibility issue is filed at:

[harvesterhci-io-github-bot]

Automation e2e test issue: harvester/tests#1407

[ibrokethecloud]

For testing the changes:

The multipath support can be validated in a kvm setup as follows:

• Create a raw disk image of atleast 250GB

• Add the same disk twice to an VM. Please ensure the following

  • The disk is marked sharable
  • A serial is provided, say disk1. The serial info needs to be same on both scsi disk additions

• Boot from the iso. Despite two paths being added to the same underlying disk, the harvester installer only shows the 1 installable disk. The serial info is important here as it is used to dedup disks with same serial info and/or wwn number

• Proceed with installation as normal, and specify the following harvester config via a config url to enable multipathd and configure multipath.conf. The additionalKernelArguments are not needed for this example to work, but have been added to help validate the arguments being passed to kernel via grub post install

  os:
  externalStorageConfig:
  enabled: true
  multiPathConfig:
  - vendor: "QEMU"
    product: "QEMU HARDDISK"
  additionalKernelArguments: "multipath=on"

• Post install of harvester, create a VM

• Shell into node and verify multipath is only showing pathing for the install disks, and longhorn disks are ignored

  multipath:~ # multipath -ll
  0QEMU_QEMU_HARDDISK_disk1 dm-0 QEMU,QEMU HARDDISK
  size=250G features='0' hwhandler='0' wp=rw
  |-+- policy='service-time 0' prio=1 status=active
  | `- 0:0:0:1 sda 8:0  active ready running
  `-+- policy='service-time 0' prio=1 status=enabled
  `- 0:0:0:2 sdb 8:16 active ready running

[noahgildersleeve]

Validated multpath with Harvester v1.4.0-rc3.Verified that multpath was running properly after install and boot and ran the p0 volumes and images test cases from automation. I don't think the one failed automated test is relevant.

I'm checking if this covers the iscsi test case as well.

I ran the test according to the quoted steps.

For testing the changes:

The multipath support can be validated in a kvm setup as follows:

• Create a raw disk image of atleast 250GB

• Add the same disk twice to an VM. Please ensure the following

  • The disk is marked sharable
  • A serial is provided, say disk1. The serial info needs to be same on both scsi disk additions

• Boot from the iso. Despite two paths being added to the same underlying disk, the harvester installer only shows the 1 installable disk. The serial info is important here as it is used to dedup disks with same serial info and/or wwn number
• Proceed with installation as normal, and specify the following harvester config via a config url to enable multipathd and configure multipath.conf. The additionalKernelArguments are not needed for this example to work, but have been added to help validate the arguments being passed to kernel via grub post install

os:
  externalStorageConfig:
    enabled: true
    multiPathConfig:
    - vendor: "QEMU"
      product: "QEMU HARDDISK"
  additionalKernelArguments: "multipath=on"

• Post install of harvester, create a VM
• Shell into node and verify multipath is only showing pathing for the install disks, and longhorn disks are ignored

multipath:~ # multipath -ll
0QEMU_QEMU_HARDDISK_disk1 dm-0 QEMU,QEMU HARDDISK
size=250G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 0:0:0:1 sda 8:0  active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 0:0:0:2 sdb 8:16 active ready running

[noahgildersleeve]

I talked with @ibrokethecloud about this and the implementation of the multpath should cover the test case. It's checking for duped serial numbers and deduping them. This should affect anything that's being passed over to the host OS from the BIOS/UEFI. This includes duped serials from RAID controllers, iSCSI, and others.

I'm going to double check this on physical hardware then close it if it works as expected.

[noahgildersleeve]

I'm closing this as fixed and will re-open it if there are any issues on bare metal.

[innobead]

Do we have a doc for this feature?

[bk201]

@ibrokethecloud we should have a doc PR before closing the issue.

[ibrokethecloud]

i will submit a docs PR with the additional config needed to leverage this

[khushboo-rancher]

Moving this to implement as this is waiting for doc PR.