search

harvester / upgrade-helpers

4 stars 11 forks source link

fix(volume): add single-replica volume check #8

Closed starbops closed 1 year ago

starbops commented 1 year ago

Add check against single-replica volumes.

related issue: harvester/harvester#4467

Test Plan

  1. Prepare a 3-node Harvester cluster

  2. Create a new StorageClass with only 1 replica

    image

  3. Create a VM with an additional volume created from the StorageClass

    image

  4. Download the check.sh script on the Harvester node and execute it; the result should resemble the following:

# On the Harvester node
$ check.sh
>>> Check all bundles ready...
All Helm bundles are ready.
>>> Check the Harvester bundle is ready...
The Harvester bundle is ready.
>>> Check all nodes are ready...
All nodes are ready.
>>> Check the CAPI cluster is provisioned...
The CAPI cluster is provisioned.
>>> Check the CAPI machines count...
CAPI machine count is equal to node count.
>>> Check the CAPI machines are running...
The CAPI machines are provisioned.
>>> Check Longhorn volumes...
checking running engine pvc-2d0763b7-3a91-439e-814f-bbee03bba48c-e-8ee4eff9...
Volume pvc-2d0763b7-3a91-439e-814f-bbee03bba48c is a single-replica volume, please consider shut down the corresponding workload before upgrading.
checking running engine pvc-4da3f21c-a19d-4058-bf2f-02c2ccd839da-e-00c90793...
Volume pvc-4da3f21c-a19d-4058-bf2f-02c2ccd839da is healthy.
checking running engine pvc-a1aecc93-ac42-4c64-a77c-91fd454507e4-e-b2411f95...
Volume pvc-a1aecc93-ac42-4c64-a77c-91fd454507e4 is healthy.
checking running engine pvc-cb2ca93e-11a4-486e-95a0-8d7404974a27-e-a2d7f563...
Volume pvc-cb2ca93e-11a4-486e-95a0-8d7404974a27 is healthy.
checking running engine pvc-cde45c42-d35e-4bd4-8933-6f9c3a51f012-e-64e07c02...
Volume pvc-cde45c42-d35e-4bd4-8933-6f9c3a51f012 is healthy.
There are volumes that need your attention!
>>> Check stale Longhorn volumes...
Checking volume longhorn-system/pvc-2d0763b7-3a91-439e-814f-bbee03bba48c...
Checking volume longhorn-system/pvc-4da3f21c-a19d-4058-bf2f-02c2ccd839da...
Checking volume longhorn-system/pvc-a1aecc93-ac42-4c64-a77c-91fd454507e4...
Checking volume longhorn-system/pvc-cb2ca93e-11a4-486e-95a0-8d7404974a27...
Checking volume longhorn-system/pvc-cde45c42-d35e-4bd4-8933-6f9c3a51f012...
There is no stale Longhorn volume.
>>> Check error pods...
All pods are OK.
All nodes have more than 30GB free space.

There are 1 failing checks!