Open W0n9 opened 3 months ago
@ibrokethecloud noticing:
Already installed harvester-seeder
+ [[ helm_v3 == \h\e\l\m\_\v\3 ]]
+ helm_v3 mapkubeapis harvester-seeder --namespace harvester-system
2024/05/17 07:20:28 Release 'harvester-seeder' will be checked for deprecated or removed Kubernetes APIs and will be updated if necessary to supported API versions.
2024/05/17 07:20:28 Get release 'harvester-seeder' latest version.
2024/05/17 07:20:28 Check release 'harvester-seeder' for deprecated or removed APIs...
2024/05/17 07:20:28 Finished checking release 'harvester-seeder' for deprecated or removed APIs.
2024/05/17 07:20:28 Release 'harvester-seeder' has no deprecated or removed APIs.
2024/05/17 07:20:28 Map of release 'harvester-seeder' deprecated or removed APIs to supported versions, completed successfully.
+ echo 'Upgrading helm_v3 chart'
+ echo 'Upgrading harvester-seeder'
+ shift 1
Upgrading harvester-seeder
+ helm_v3 upgrade --repo http://harvester-cluster-repo.cattle-system.svc/charts --version 0.1.0 harvester-seeder harvester-seeder --values /config/values-01_HelmChart.yaml
Error: chart "harvester-seeder" version "0.1.0" not found in http://harvester-cluster-repo.cattle-system.svc/charts repository
Specifically, the Error: chart "harvester-seeder" version "0.1.0" not found in http://harvester-cluster-repo.cattle-system.svc/charts repository
about 6/7 times in: harvester-system.helm-install-harvester-seeder-9gccc.helm.20240517.log
(extracted from .gz)
But the pre-drain on dc-0 looks to be successful:
nvim harvester-system.hvst-upgrade-hjw2p-pre-drain-harvester-dc-0-xsdr2.apply.20240517.log
With the tail-end being:
Volume pvc-bf6c793f-7597-4dcb-9842-a9e862d05d72 is healthy.
Checking running engine pvc-fd1960b3-ef2c-4bcb-9c45-72322945648a-e-ec25015f...
+ echo Checking running engine pvc-fd1960b3-ef2c-4bcb-9c45-72322945648a-e-ec25015f...
+ '[' true ']'
+ '[' 4 -gt 2 ']'
++ kubectl get volumes.longhorn.io/pvc-fd1960b3-ef2c-4bcb-9c45-72322945648a -n longhorn-system -o 'jsonpath={.status.robustness}'
+ robustness=healthy
+ '[' healthy = healthy ']'
+ echo 'Volume pvc-fd1960b3-ef2c-4bcb-9c45-72322945648a is healthy.'
Volume pvc-fd1960b3-ef2c-4bcb-9c45-72322945648a is healthy.
+ break
+ read -r lh_engine lh_volume
+ upgrade-helper vm-live-migrate-detector harvester-dc-0 --shutdown
time="2024-05-17T07:14:02Z" level=info msg="Starting VM Live Migrate Detector"
time="2024-05-17T07:14:02Z" level=info msg="Non-migratable VM(s): []"
+ kubectl taint node harvester-dc-0 --overwrite kubevirt.io/drain=draining:NoSchedule
node/harvester-dc-0 modified
+ wait_vms_out_or_shutdown
+ local vm_count
+ '[' true ']'
++ get_running_vm_count
++ local vmioutput=/tmp/vmioutput.yaml
++ rm -f /tmp/vmioutput.yaml
++ local EXIT_CODE=0
++ kubectl get vmi -A -l kubevirt.io/nodeName=harvester-dc-0 -o yaml
++ [[ 0 -eq 0 ]]
+++ yq e '.items | map(select(.status.phase!="Succeeded")) | length' /tmp/vmioutput.yaml
++ local count=0
++ echo 0
++ rm -f /tmp/vmioutput.yaml
+ vm_count=0
+ '[' 0 = 0 ']'
+ break
+ echo 'all VMs on node harvester-dc-0 have been live-migrated or shutdown'
+ wait_evacuation_pdb_gone
all VMs on node harvester-dc-0 have been live-migrated or shutdown
+ kubectl get pdb -o name -A
+ grep kubevirt-migration-pdb-kubevirt-evacuation-
+ patch_logging_event_audit
+ detect_upgrade
++ kubectl get upgrades.harvesterhci.io hvst-upgrade-hjw2p -n harvester-system -o yaml
+ upgrade_obj='apiVersion: harvesterhci.io/v1beta1
Hi @W0n9, Could you also generate a support bundle for investigation? Ref: https://docs.harvesterhci.io/v1.2/troubleshooting/harvester/#generate-a-support-bundle
Here is my support bundle @Vicente-Cheng
Yesterday, after upgrade failed: supportbundle_7e8aefe3-479a-4fa4-97d8-af2f283e4577_2024-05-17T09-01-49Z.001.zip supportbundle_7e8aefe3-479a-4fa4-97d8-af2f283e4577_2024-05-17T09-01-49Z.002.zip
Today, after some fix but failed again: supportbundle_7e8aefe3-479a-4fa4-97d8-af2f283e4577_2024-05-18T02-54-16Z.001.zip supportbundle_7e8aefe3-479a-4fa4-97d8-af2f283e4577_2024-05-18T02-54-16Z.002.zip
Hi @W0n9,
Could you also provide the command to unzip this? I use 7-zip, but it looks like it is not correct.
If the SB is too large to upload, you could send the bundle to harvester-support-bundle@suse.com with this issue ID. Thanks!
Hi, @Vicente-Cheng
You can try to rename it from file_name.001.zip to file_name.zip.001 .
I have sent e-mail to you.
@starbops Please help check this issue, thanks.
Hi @W0n9, I couldn't find any YAML files in the support bundle files you provided, so it's hard to determine the cluster's precise state. Did you ever change the support-bundle-namespaces
setting before generating the support bundle file?
Under the Advanced -> Settings page of the dashboard:
Please also help provide the Upgrade and Setting CRs. You could run the following commands and get the output:
kubectl -n harvester-system get upgrades.harvesterhci -o yaml
kubectl get settings.harvesterhci -o yaml
From the rancher logs (supportbundle_7e8aefe3-479a-4fa4-97d8-af2f283e4577_2024-05-17T09-01-49Z/logs/cattle-system/rancher-654c45c575-zrmkv/rancher.log
) we found the following line was repeated again and again:
024-05-17T08:55:00.616288694Z 2024/05/17 08:55:00 [INFO] [planner] rkecluster fleet-local/local: configuring bootstrap node(s) custom-fb4dda4acb64: waiting for probes: kube-controller-manager, kube-scheduler
It might be related to certificates expiration. Perhaps you could check if this works for you. Thanks.
Hi, @starbops
I never change the support-bundle-namespaces
and keep it be default.
Here is the YAML:
apiVersion: v1
items:
- apiVersion: harvesterhci.io/v1beta1
kind: Upgrade
metadata:
annotations:
harvesterhci.io/replica-replenishment-wait-interval: "600"
creationTimestamp: "2023-09-09T06:41:47Z"
finalizers:
- wrangler.cattle.io/harvester-upgrade-controller
generateName: hvst-upgrade-
generation: 41
labels:
harvesterhci.io/read-message: "true"
harvesterhci.io/upgradeState: Succeeded
name: hvst-upgrade-f47wk
namespace: harvester-system
resourceVersion: "290443078"
uid: 4382668b-352c-4263-a827-806f60e5b4c7
spec:
image: ""
logEnabled: true
version: v1.2.0
status:
conditions:
- lastUpdateTime: "2023-09-09T11:52:45Z"
status: "True"
type: Completed
- lastUpdateTime: "2023-09-09T06:42:07Z"
status: "True"
type: LogReady
- lastUpdateTime: "2023-09-09T06:44:00Z"
status: "True"
type: ImageReady
- lastUpdateTime: "2023-09-09T06:46:24Z"
status: "True"
type: RepoReady
- lastUpdateTime: "2023-09-09T07:21:42Z"
status: "True"
type: NodesPrepared
- lastUpdateTime: "2023-09-09T10:25:23Z"
status: "True"
type: SystemServicesUpgraded
- lastUpdateTime: "2023-09-09T11:52:45Z"
status: "True"
type: NodesUpgraded
imageID: harvester-system/harvester-iso-7x6wp
nodeStatuses:
harvester-dc-0:
state: Succeeded
harvester-dc-1:
state: Succeeded
harvester-dc-2:
state: Succeeded
harvester-dc-3:
state: Succeeded
previousVersion: v1.1.2
repoInfo: |
release:
harvester: v1.2.0
harvesterChart: 1.2.0
os: Harvester v1.2.0
kubernetes: v1.25.9+rke2r1
rancher: v2.7.5
monitoringChart: 102.0.0+up40.1.2
minUpgradableVersion: v1.1.2
- apiVersion: harvesterhci.io/v1beta1
kind: Upgrade
metadata:
annotations:
harvesterhci.io/replica-replenishment-wait-interval: "600"
creationTimestamp: "2024-05-17T06:39:35Z"
finalizers:
- wrangler.cattle.io/harvester-upgrade-controller
generateName: hvst-upgrade-
generation: 22
labels:
harvesterhci.io/latestUpgrade: "true"
harvesterhci.io/upgradeState: UpgradingNodes
name: hvst-upgrade-hjw2p
namespace: harvester-system
resourceVersion: "599965758"
uid: 7e32dcce-8f25-4d7d-bd86-fa16c63737f1
spec:
image: ""
logEnabled: true
version: v1.2.2
status:
conditions:
- status: Unknown
type: Completed
- lastUpdateTime: "2024-05-17T06:40:00Z"
status: "True"
type: LogReady
- lastUpdateTime: "2024-05-17T06:41:04Z"
status: "True"
type: ImageReady
- lastUpdateTime: "2024-05-17T06:43:48Z"
status: "True"
type: RepoReady
- lastUpdateTime: "2024-05-17T06:59:11Z"
status: "True"
type: NodesPrepared
- lastUpdateTime: "2024-05-17T07:13:55Z"
status: "True"
type: SystemServicesUpgraded
- status: Unknown
type: NodesUpgraded
imageID: harvester-system/hvst-upgrade-hjw2p
nodeStatuses:
harvester-dc-0:
state: Pre-drained
harvester-dc-1:
state: Images preloaded
harvester-dc-2:
state: Images preloaded
harvester-dc-3:
state: Images preloaded
previousVersion: v1.2.1
repoInfo: |
release:
harvester: v1.2.2
harvesterChart: 1.2.2
os: Harvester v1.2.2
kubernetes: v1.26.15+rke2r1
rancher: v2.8.2
monitoringChart: 103.0.3+up45.31.1
minUpgradableVersion: v1.2.1
upgradeLog: hvst-upgrade-hjw2p-upgradelog
- apiVersion: harvesterhci.io/v1beta1
kind: Upgrade
metadata:
annotations:
harvesterhci.io/replica-replenishment-wait-interval: "600"
creationTimestamp: "2023-04-25T13:56:45Z"
finalizers:
- wrangler.cattle.io/harvester-upgrade-controller
generateName: hvst-upgrade-
generation: 31
labels:
harvesterhci.io/read-message: "true"
harvesterhci.io/upgradeState: Succeeded
name: hvst-upgrade-nszm9
namespace: harvester-system
resourceVersion: "217748009"
uid: 23f16027-63e5-464d-8332-22b187f9ee14
spec:
image: ""
logEnabled: true
version: v1.1.2
status:
conditions:
- lastUpdateTime: "2023-04-25T15:35:16Z"
status: "True"
type: Completed
- lastUpdateTime: "2023-04-25T14:00:58Z"
status: "True"
type: ImageReady
- lastUpdateTime: "2023-04-25T14:04:05Z"
status: "True"
type: RepoReady
- lastUpdateTime: "2023-04-25T14:36:40Z"
status: "True"
type: NodesPrepared
- lastUpdateTime: "2023-04-25T14:47:01Z"
status: "True"
type: SystemServicesUpgraded
- lastUpdateTime: "2023-04-25T15:35:16Z"
status: "True"
type: NodesUpgraded
imageID: harvester-system/harvester-iso-wjfvr
nodeStatuses:
harvester-dc-0:
state: Succeeded
harvester-dc-1:
state: Succeeded
harvester-dc-2:
state: Succeeded
previousVersion: v1.1.1
repoInfo: |
release:
harvester: v1.1.2
harvesterChart: 1.1.2
os: Harvester v1.1.2
kubernetes: v1.24.11+rke2r1
rancher: v2.6.11
monitoringChart: 100.1.0+up19.0.3
minUpgradableVersion: v1.1.0
- apiVersion: harvesterhci.io/v1beta1
kind: Upgrade
metadata:
annotations:
harvesterhci.io/replica-replenishment-wait-interval: "600"
creationTimestamp: "2023-10-27T06:56:30Z"
finalizers:
- wrangler.cattle.io/harvester-upgrade-controller
generateName: hvst-upgrade-
generation: 41
labels:
harvesterhci.io/read-message: "true"
harvesterhci.io/upgradeState: Succeeded
name: hvst-upgrade-wmxxs
namespace: harvester-system
resourceVersion: "599919539"
uid: 4534471a-5492-45bc-9d30-4147ce7c1cb4
spec:
image: ""
logEnabled: true
version: v1.2.1
status:
conditions:
- lastUpdateTime: "2023-10-27T09:47:15Z"
status: "True"
type: Completed
- lastUpdateTime: "2023-10-27T06:56:57Z"
status: "True"
type: LogReady
- lastUpdateTime: "2023-10-27T06:58:48Z"
status: "True"
type: ImageReady
- lastUpdateTime: "2023-10-27T07:02:28Z"
status: "True"
type: RepoReady
- lastUpdateTime: "2023-10-27T07:10:38Z"
status: "True"
type: NodesPrepared
- lastUpdateTime: "2023-10-27T07:11:49Z"
status: "True"
type: SystemServicesUpgraded
- lastUpdateTime: "2023-10-27T09:47:15Z"
status: "True"
type: NodesUpgraded
imageID: harvester-system/hvst-upgrade-wmxxs
nodeStatuses:
harvester-dc-0:
state: Succeeded
harvester-dc-1:
state: Succeeded
harvester-dc-2:
state: Succeeded
harvester-dc-3:
state: Succeeded
previousVersion: v1.2.0
repoInfo: |
release:
harvester: v1.2.1
harvesterChart: 1.2.1
os: Harvester v1.2.1
kubernetes: v1.25.9+rke2r1
rancher: v2.7.5
monitoringChart: 102.0.0+up40.1.2
minUpgradableVersion: v1.1.2
kind: List
metadata:
resourceVersion: ""
apiVersion: v1
items:
- apiVersion: harvesterhci.io/v1beta1
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: additional-ca
resourceVersion: "8451"
uid: 8b9493e6-d17c-4e12-a0c4-e52bf1bea2f2
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: 1.1.9
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: api-ui-version
resourceVersion: "8448"
uid: 969dc386-9401-403d-9373-10efa8fa73b7
status: {}
value: 1.1.9
- apiVersion: harvesterhci.io/v1beta1
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: auto-disk-provision-paths
resourceVersion: "8469"
uid: e7556dd1-af28-4b8e-a3a9-b9d289d9e7d0
status: {}
- apiVersion: harvesterhci.io/v1beta1
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 29
name: backup-target
resourceVersion: "339358074"
uid: 79121a2e-fef6-422f-8510-866eaf575847
status:
conditions:
- lastUpdateTime: "2023-11-27T09:58:03Z"
status: "True"
type: configured
value: '{"type":"s3","endpoint":"https://s3-east-internal.buct.edu.cn:9000","accessKeyId":"","secretAccessKey":"","bucketName":"vm-backups","bucketRegion":"bj","cert":"","virtualHostedStyle":false}'
- apiVersion: harvesterhci.io/v1beta1
kind: Setting
metadata:
annotations:
harvesterhci.io/hash: 30cff57968b3b923f62143d6987eb1eb077ca0d8061b21f0db658123
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 3
name: cluster-registration-url
resourceVersion: "14406"
uid: 5b7684de-3054-4cec-beec-6d6265d89bd4
status:
conditions:
- lastUpdateTime: "2023-04-25T07:23:29Z"
status: "True"
type: configured
value: https://rancher.buct.edu.cn/v3/import/td7vhx6j6sg6qxp88g5flhzzsdndz96xjrl7zvd2nzq2hwkp5b788t_c-m-76mpd8w7.yaml
- apiVersion: harvesterhci.io/v1beta1
kind: Setting
metadata:
annotations:
harvesterhci.io/upgrade-patched: v1.2.2
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: containerd-registry
resourceVersion: "599960369"
uid: de373ce8-27d0-4004-8947-609e0f1f0315
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: '{"driver.longhorn.io":{"volumeSnapshotClassName":"longhorn-snapshot","backupVolumeSnapshotClassName":"longhorn"}}'
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: csi-driver-config
resourceVersion: "8454"
uid: a7fd421c-5e71-42c0-ad13-2e268f6b1a16
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: longhorn
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: default-storage-class
resourceVersion: "8460"
uid: a6f605fb-8789-4570-aa68-ee8870da980c
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: "120"
kind: Setting
metadata:
creationTimestamp: "2023-09-09T07:25:51Z"
generation: 1
name: default-vm-termination-grace-period-seconds
resourceVersion: "217801286"
uid: e488e5d9-a0f7-41ff-922e-2b7c1dfb593f
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: '{"harvester-cloud-provider":">=0.0.1 <0.3.0","harvester-csi-provider":">=0.0.1
<0.3.0"}'
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 3
name: harvester-csi-ccm-versions
resourceVersion: "217801288"
uid: f9926d02-cf54-4b64-8662-dc13109361eb
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: '{}'
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: http-proxy
resourceVersion: "8425"
uid: 322b0282-a4b9-4262-8095-e3e9185bc4a1
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: info
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: log-level
resourceVersion: "8457"
uid: 9c4e74df-9f16-4249-9fd0-da88f8932763
status: {}
- apiVersion: harvesterhci.io/v1beta1
kind: Setting
metadata:
creationTimestamp: "2023-09-09T07:25:51Z"
generation: 1
name: ntp-servers
resourceVersion: "217801281"
uid: 3cf5d828-79b5-484e-abdb-4482a1d961c9
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: '{"cpu":1600,"memory":150,"storage":200}'
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: overcommit-config
resourceVersion: "8423"
uid: 1f0fdcf1-1e50-4fc0-abd3-646f3fe0d018
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: https://releases.rancher.com/harvester
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: release-download-url
resourceVersion: "8459"
uid: 4ba73310-bcd8-4c2c-84c6-af147d4058fb
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: dev
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 5
name: server-version
resourceVersion: "599949818"
uid: d1cbce86-a2d6-4bb4-b73a-caf15630c82b
status: {}
value: v1.2.2
- apiVersion: harvesterhci.io/v1beta1
default: '{}'
kind: Setting
metadata:
annotations:
harvesterhci.io/hash: d14a028c2a3a2bc9476102bb288234c415a2b01f828ea62ac5b3e42f
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 2
name: ssl-certificates
resourceVersion: "8498"
uid: 514e3aeb-5e62-4b0c-ac33-cb8a2a399182
status:
conditions:
- lastUpdateTime: "2023-04-25T07:17:36Z"
status: "False"
type: configured
- apiVersion: harvesterhci.io/v1beta1
default: '{}'
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: ssl-parameters
resourceVersion: "8463"
uid: da2eeb6e-0340-4aaf-85a1-e1079b2e9d51
status: {}
- apiVersion: harvesterhci.io/v1beta1
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: storage-network
resourceVersion: "8455"
uid: 9ee25fee-5bbb-4a31-9663-204d04c090db
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: '{"repository":"rancher/support-bundle-kit","tag":"v0.0.37","imagePullPolicy":"IfNotPresent"}'
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 66
name: support-bundle-image
resourceVersion: "599953683"
uid: 9554026e-23ce-45b8-935a-e4146a7278a9
status: {}
- apiVersion: harvesterhci.io/v1beta1
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: support-bundle-namespaces
resourceVersion: "8461"
uid: 886c947d-dd8e-46a9-a934-ecd0282900c9
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: "10"
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: support-bundle-timeout
resourceVersion: "8442"
uid: 357d73d0-7e6d-4b60-a3bf-962a7d483b49
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: https://releases.rancher.com/harvester-ui/dashboard/release-harvester-v1.2/index.html
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 2
name: ui-index
resourceVersion: "217801285"
uid: 237c20d9-09c8-4d43-a38b-12f5860cd02d
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: /usr/share/harvester/harvester
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: ui-path
resourceVersion: "8429"
uid: 4e838361-c370-48fd-b86d-f3b3bb799c43
status: {}
value: /usr/share/harvester/harvester
- apiVersion: harvesterhci.io/v1beta1
default: https://releases.rancher.com/harvester-ui/plugin/harvester-release-harvester-v1.2/harvester-release-harvester-v1.2.umd.min.js
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 2
name: ui-plugin-index
resourceVersion: "217801290"
uid: 8270d656-502f-4f73-b778-5b7ae82687ca
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: auto
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: ui-source
resourceVersion: "8470"
uid: 839901a8-0944-474c-ac21-7469438719ef
status: {}
- apiVersion: harvesterhci.io/v1beta1
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: upgradable-versions
resourceVersion: "8462"
uid: 84d85c47-5057-415b-8ba1-7040a49ae09e
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: "true"
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: upgrade-checker-enabled
resourceVersion: "8432"
uid: 2d2b4c3d-8a23-40df-bab6-edc60a62368c
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: https://harvester-upgrade-responder.rancher.io/v1/checkupgrade
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: upgrade-checker-url
resourceVersion: "8437"
uid: fa4e35fe-d384-48df-ba77-8cb12c175b9e
status: {}
- apiVersion: harvesterhci.io/v1beta1
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: vip-pools
resourceVersion: "8453"
uid: 4b1dedb7-9e36-4a07-a37f-606db62aac86
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: '{"enable":true,"period":300}'
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: vm-force-reset-policy
resourceVersion: "8452"
uid: 5fae2e04-cd67-4214-848d-9cd21b1b1fd4
status: {}
- apiVersion: harvesterhci.io/v1beta1
default: longhorn
kind: Setting
metadata:
creationTimestamp: "2023-04-25T07:17:35Z"
generation: 1
name: volume-snapshot-class
resourceVersion: "8450"
uid: 9616bfa4-5c17-42eb-84ce-32d84238401c
status: {}
kind: List
metadata:
resourceVersion: ""
Hi @W0n9, Could you also check the suggestion from @starbops
It might be related to certificates expiration. Perhaps you could check if https://github.com/harvester/harvester/issues/3863#issuecomment-1539681311 works for you. Thanks.
Hi @Vicente-Cheng , I have tried https://github.com/harvester/harvester/issues/3863#issuecomment-1539681311 in my 2 long-running harvester cluster, and it works. Thanks
cert rotate is implemented in v1.3.0, @starbops can you help add this (https://docs.harvesterhci.io/v1.3/upgrade/v1-1-2-to-v1-2-0#7-upgrade-stuck-in-the-upgrading-system-service-state) and seeder one to the known issue sections of 1.2.2 upgrade page? Thanks.
Describe the bug
To Reproduce Steps to reproduce the behavior:
Expected behavior
Cluster should be upgraded successfully
Support bundle
hvst-upgrade-hjw2p-upgradelog-archive-2024-05-17T07-59-36Z.zip
Environment
Additional context Add any other context about the problem here.