valaparthvi
commented
2 months ago EKS Provisioning tests
``` make e2e-k8s-chart-support-provisioning-tests-upgrade go install -mod=mod github.com/onsi/ginkgo/v2/ginkgo go install -mod=mod github.com/onsi/gomega go mod tidy ginkgo -v -r --timeout=3h --keep-going --randomize-all --randomize-suites --focus "K8sChartSupportUpgradeProvisioning" ./hosted/eks/k8s_chart_support/upgrade Running Suite: K8sChartSupportUpgrade Suite - /home/pvala/go/src/github.com/rancher/hosted-providers-e2e/hosted/eks/k8s_chart_support/upgrade ============================================================================================================================================= Random Seed: 1726672644 - will randomize all specs Will run 1 of 2 specs ------------------------------ K8sChartSupportUpgradeProvisioning should successfully test k8s chart support provisioning in an upgrade scenario /home/pvala/go/src/github.com/rancher/hosted-providers-e2e/hosted/eks/k8s_chart_support/upgrade/k8s_chart_support_provisioning_upgrade_test.go:31 STEP: Adding the necessary chart repos @ 09/18/24 20:47:29.48 STEP: Installing Rancher Manager v2.9.0 @ 09/18/24 20:47:29.528 "level"=0 "msg"="Deploying rancher: [upgrade --install rancher rancher-latest/rancher --namespace cattle-system --create-namespace --set hostname=ec2-65-0-56-221.ap-south-1.compute.amazonaws.com --set bootstrapPassword=adminadmin --set replicas=1 --set global.cattle.psp.enabled=false --wait --version 2.9.0]" "level"=0 "msg"="Using Common SynchronizedBeforeSuite ..." "level"=0 "msg"="Using Common BeforeSuite ..." "level"=0 "msg"="Using EKS version 1.29 for cluster auto-eks-hp-ci-xuhey" time="2024-09-18T21:01:31+05:30" level=info msg="Cluster status is active!" "level"=0 "msg"="Testing K8s 1.30 chart support for provisioning on Rancher upgraded from 2.9.0 to devel/2.9" STEP: checking cluster name is same @ 09/18/24 21:01:31.641 STEP: checking service account token secret @ 09/18/24 21:01:31.641 time="2024-09-18T21:01:31+05:30" level=info msg="serviceAccountTokenSecret in this cluster is: cluster-serviceaccounttoken-ctwpx" STEP: checking all management nodes are ready @ 09/18/24 21:01:31.78 STEP: checking all pods are ready @ 09/18/24 21:01:31.981 time="2024-09-18T21:01:31+05:30" level=info msg="All nodes in the cluster are in an active state!" STEP: checking the chart version @ 09/18/24 21:01:57.534 "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 15:09:15.261222011 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 15:09:09.515356607 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"}]\n" "level"=0 "msg"="Original chart version: 104.2.0+up1.9.2" STEP: upgrading rancher @ 09/18/24 21:01:57.801 "level"=0 "msg"="Deploying rancher: [upgrade --install rancher rancher-latest/rancher --namespace cattle-system --create-namespace --set hostname=ec2-65-0-56-221.ap-south-1.compute.amazonaws.com --set bootstrapPassword=adminadmin --set replicas=1 --set global.cattle.psp.enabled=false --wait --devel --set rancherImageTag=v2.9-head]" STEP: ensuring operator pods are also up @ 09/18/24 21:04:37.296 STEP: regenerating the token and initiating a new rancher client @ 09/18/24 21:04:37.858 STEP: making sure the local cluster is ready @ 09/18/24 21:04:39.387 STEP: checking all management nodes are ready @ 09/18/24 21:04:39.387 STEP: checking all pods are ready @ 09/18/24 21:04:39.451 time="2024-09-18T21:04:39+05:30" level=info msg="All nodes in the cluster are in an active state!" STEP: checking the chart version and validating it is > the old version @ 09/18/24 21:04:44.805 "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 15:09:15.261222011 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 15:09:09.515356607 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"}]\n" "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 15:09:15.261222011 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 15:09:09.515356607 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"}]\n" "level"=0 "msg"="Upgraded chart version: 104.2.0+up1.9.2" STEP: fetching a list of available k8s versions and ensure the v1.30 is present in the list and upgrading the cluster to it @ 09/18/24 21:04:45.345 "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." STEP: upgrading the NodeGroups @ 09/18/24 21:15:47.048 time="2024-09-18T21:15:47+05:30" level=info msg="ClusterID: c-cx48r, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" time="2024-09-18T21:15:51+05:30" level=info msg="ClusterID: c-cx48r, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T21:15:51+05:30" level=info msg="ClusterID: c-cx48r, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T21:25:31+05:30" level=info msg="ClusterID: c-cx48r, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" "level"=0 "msg"="waiting for the nodegroup upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="waiting for the nodegroup upgrade to appear in EKSStatus.UpstreamSpec ..." STEP: fetching a value to downgrade to @ 09/18/24 21:26:02.038 "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 15:09:15.261222011 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 15:09:09.515356607 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"}]\n" "level"=0 "msg"="[{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"104.1.0+up1.9.1\",\"app_version\":\"1.9.1\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"104.0.1+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"104.0.0+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"103.4.0+up1.3.4\",\"app_version\":\"1.3.4\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"103.3.0+up1.3.3\",\"app_version\":\"1.3.3\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"103.2.0+up1.3.2\",\"app_version\":\"1.3.2\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"103.1.0+up1.3.1\",\"app_version\":\"1.3.1\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"103.0.0+up1.3.0\",\"app_version\":\"1.3.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"102.2.0+up1.2.3\",\"app_version\":\"1.2.3\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"102.1.4+up1.2.2\",\"app_version\":\"1.2.2\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"102.1.1+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"102.1.0+up1.2.1\",\"app_version\":\"1.2.1\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"102.0.2+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"102.0.1+up1.1.5\",\"app_version\":\"1.1.5\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"102.0.0+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"101.0.0+up1.1.5\",\"app_version\":\"1.1.5\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"104.1.0+up1.9.1\",\"app_version\":\"1.9.1\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"104.0.1+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"104.0.0+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"103.4.0+up1.3.4\",\"app_version\":\"1.3.4\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"103.3.0+up1.3.3\",\"app_version\":\"1.3.3\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"103.2.0+up1.3.2\",\"app_version\":\"1.3.2\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"103.1.0+up1.3.1\",\"app_version\":\"1.3.1\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"103.0.0+up1.3.0\",\"app_version\":\"1.3.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"102.2.0+up1.2.3\",\"app_version\":\"1.2.3\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"102.1.4+up1.2.2\",\"app_version\":\"1.2.2\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"102.1.1+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"102.1.0+up1.2.1\",\"app_version\":\"1.2.1\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"102.0.2+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"102.0.1+up1.1.5\",\"app_version\":\"1.1.5\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"102.0.0+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"101.0.0+up1.1.5\",\"app_version\":\"1.1.5\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"104.1.0+up1.9.1\",\"app_version\":\"1.9.1\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"104.0.1+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"104.0.0+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"103.4.0+up1.3.4\",\"app_version\":\"1.3.4\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"103.3.0+up1.3.3\",\"app_version\":\"1.3.3\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"103.2.0+up1.3.2\",\"app_version\":\"1.3.2\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"103.1.0+up1.3.1\",\"app_version\":\"1.3.1\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"103.0.0+up1.3.0\",\"app_version\":\"1.3.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"102.2.0+up1.2.3\",\"app_version\":\"1.2.3\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"102.1.4+up1.2.2\",\"app_version\":\"1.2.2\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"102.1.1+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"102.1.0+up1.2.1\",\"app_version\":\"1.2.1\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"102.0.2+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"102.0.1+up1.1.5\",\"app_version\":\"1.1.5\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"102.0.0+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"101.0.0+up1.1.5\",\"app_version\":\"1.1.5\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"104.1.0+up1.9.1\",\"app_version\":\"1.9.1\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"104.0.1+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"104.0.0+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"103.4.0+up1.3.4\",\"app_version\":\"1.3.4\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"103.3.0+up1.3.3\",\"app_version\":\"1.3.3\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"103.2.0+up1.3.2\",\"app_version\":\"1.3.2\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"103.1.0+up1.3.1\",\"app_version\":\"1.3.1\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"103.0.0+up1.3.0\",\"app_version\":\"1.3.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"102.2.0+up1.2.3\",\"app_version\":\"1.2.3\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"102.1.4+up1.2.2\",\"app_version\":\"1.2.2\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"102.1.1+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"102.1.0+up1.2.1\",\"app_version\":\"1.2.1\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"102.0.2+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"102.0.1+up1.1.5\",\"app_version\":\"1.1.5\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"102.0.0+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"101.0.0+up1.1.5\",\"app_version\":\"1.1.5\",\"description\":\"EKS Operator CustomResourceDefinitions\"}]\n" STEP: downgrading the chart version @ 09/18/24 21:26:02.604 "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 15:09:15.261222011 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 15:09:09.515356607 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"}]\n" "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 15:09:15.261222011 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 15:09:09.515356607 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"}]\n" "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 21:26:04.091761544 +0530 IST\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.1.0+up1.9.1\",\"app_version\":\"1.9.1\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 21:26:07.768175682 +0530 IST\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.1.0+up1.9.1\",\"app_version\":\"1.9.1\"}]\n" STEP: making a change to the cluster (scaling the node up) to validate functionality after chart downgrade @ 09/18/24 21:26:08.431 time="2024-09-18T21:26:08+05:30" level=info msg="ClusterID: c-cx48r, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" time="2024-09-18T21:26:09+05:30" level=info msg="ClusterID: c-cx48r, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T21:26:09+05:30" level=info msg="ClusterID: c-cx48r, Message: [operation error EKS: UpdateNodegroupConfig, https response error StatusCode: 409, RequestID: dc671bdb-7b0b-4ad3-a7dd-b0c14a526206, ResourceInUseException: Nodegroup cannot be updated as it is currently not in Active State], Error: true, State: updating, Transitioning: false" time="2024-09-18T21:26:14+05:30" level=info msg="ClusterID: c-cx48r, Message: [], Error: false, State: updating, Transitioning: true" "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." time="2024-09-18T21:26:44+05:30" level=info msg="ClusterID: c-cx48r, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." STEP: uninstalling the operator chart @ 09/18/24 21:30:45.505 "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 21:26:04.091761544 +0530 IST\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.1.0+up1.9.1\",\"app_version\":\"1.9.1\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 21:26:07.768175682 +0530 IST\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.1.0+up1.9.1\",\"app_version\":\"1.9.1\"}]\n" STEP: making a change(adding a nodepool) to the cluster to re-install the operator and validating it is re-installed to the latest/upgraded version @ 09/18/24 21:30:47.116 STEP: ensuring that the chart is re-installed to the latest/upgraded version @ 09/18/24 21:30:47.23 "level"=0 "msg"="[]\n" "level"=0 "msg"="[{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 16:00:52.311453025 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"}]\n" time="2024-09-18T21:30:53+05:30" level=info msg="ClusterID: c-cx48r, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" time="2024-09-18T21:32:05+05:30" level=info msg="ClusterID: c-cx48r, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T21:32:05+05:30" level=info msg="ClusterID: c-cx48r, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T21:33:11+05:30" level=info msg="ClusterID: c-cx48r, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" STEP: Installing Rancher back to its original version 2.9.0 @ 09/18/24 21:35:41.067 "level"=0 "msg"="Deploying rancher: [upgrade --install rancher rancher-latest/rancher --namespace cattle-system --create-namespace --set hostname=ec2-65-0-56-221.ap-south-1.compute.amazonaws.com --set bootstrapPassword=adminadmin --set replicas=1 --set global.cattle.psp.enabled=false --wait --version 2.9.0]" STEP: Uninstalling the existing operator charts @ 09/18/24 21:38:20.774 "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 16:00:58.34201514 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 16:00:52.311453025 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"}]\n" • [3052.467 seconds] ------------------------------ S Ran 1 of 2 Specs in 3052.467 seconds SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 1 Skipped PASS ```
EKS Import tests
``` make e2e-k8s-chart-support-import-tests-upgrade go install -mod=mod github.com/onsi/ginkgo/v2/ginkgo go install -mod=mod github.com/onsi/gomega go mod tidy ginkgo -v -r --timeout=3h --keep-going --randomize-all --randomize-suites --focus "K8sChartSupportUpgradeImport" ./hosted/eks/k8s_chart_support/upgrade Running Suite: K8sChartSupportUpgrade Suite - /home/pvala/go/src/github.com/rancher/hosted-providers-e2e/hosted/eks/k8s_chart_support/upgrade ============================================================================================================================================= Random Seed: 1726667624 - will randomize all specs Will run 1 of 2 specs S ------------------------------ K8sChartSupportUpgradeImport should successfully test k8s chart support import in an upgrade scenario /home/pvala/go/src/github.com/rancher/hosted-providers-e2e/hosted/eks/k8s_chart_support/upgrade/k8s_chart_support_import_upgrade_test.go:35 STEP: Adding the necessary chart repos @ 09/18/24 19:23:49.96 STEP: Installing Rancher Manager v2.9.0 @ 09/18/24 19:23:50.008 "level"=0 "msg"="Deploying rancher: [upgrade --install rancher rancher-latest/rancher --namespace cattle-system --create-namespace --set hostname=ec2-65-0-56-221.ap-south-1.compute.amazonaws.com --set bootstrapPassword=adminadmin --set replicas=1 --set global.cattle.psp.enabled=false --wait --version 2.9.0]" "level"=0 "msg"="Using Common SynchronizedBeforeSuite ..." "level"=0 "msg"="Using Common BeforeSuite ..." "level"=0 "msg"="Using EKS version 1.29 for cluster auto-eks-hp-ci-vfgqt" time="2024-09-18T19:26:45+05:30" level=info msg="Cluster status is active!" "level"=0 "msg"="Testing K8s 1.30 chart support for import on Rancher upgraded from 2.9.0 to devel/2.9" STEP: checking cluster name is same @ 09/18/24 19:26:45.582 STEP: checking service account token secret @ 09/18/24 19:26:45.582 STEP: checking all management nodes are ready @ 09/18/24 19:26:45.709 time="2024-09-18T19:26:45+05:30" level=info msg="serviceAccountTokenSecret in this cluster is: cluster-serviceaccounttoken-glsnx" time="2024-09-18T19:26:45+05:30" level=info msg="All nodes in the cluster are in an active state!" STEP: checking all pods are ready @ 09/18/24 19:26:45.885 STEP: checking the chart version @ 09/18/24 19:27:01.268 "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 13:44:49.689564112 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 13:44:57.760335064 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"}]\n" "level"=0 "msg"="Original chart version: 104.2.0+up1.9.2" STEP: upgrading rancher @ 09/18/24 19:27:01.591 "level"=0 "msg"="Deploying rancher: [upgrade --install rancher rancher-latest/rancher --namespace cattle-system --create-namespace --set hostname=ec2-65-0-56-221.ap-south-1.compute.amazonaws.com --set bootstrapPassword=adminadmin --set replicas=1 --set global.cattle.psp.enabled=false --wait --devel --set rancherImageTag=v2.9-head]" STEP: ensuring operator pods are also up @ 09/18/24 19:29:42.408 STEP: regenerating the token and initiating a new rancher client @ 09/18/24 19:29:43 STEP: making sure the local cluster is ready @ 09/18/24 19:29:44.554 STEP: checking all management nodes are ready @ 09/18/24 19:29:44.554 STEP: checking all pods are ready @ 09/18/24 19:29:44.618 time="2024-09-18T19:29:44+05:30" level=info msg="All nodes in the cluster are in an active state!" STEP: checking the chart version and validating it is > the old version @ 09/18/24 19:29:49.961 "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 13:44:49.689564112 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 13:44:57.760335064 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"}]\n" "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 13:44:49.689564112 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 13:44:57.760335064 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"}]\n" "level"=0 "msg"="Upgraded chart version: 104.2.0+up1.9.2" STEP: fetching a list of available k8s versions and ensure the v1.30 is present in the list and upgrading the cluster to it @ 09/18/24 19:29:50.513 "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for k8s upgrade to appear in EKSStatus.UpstreamSpec ..." STEP: fetching a value to downgrade to @ 09/18/24 19:41:52.455 "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 13:44:49.689564112 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 13:44:57.760335064 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"}]\n" "level"=0 "msg"="[{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"104.1.0+up1.9.1\",\"app_version\":\"1.9.1\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"104.0.1+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"104.0.0+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"103.4.0+up1.3.4\",\"app_version\":\"1.3.4\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"103.3.0+up1.3.3\",\"app_version\":\"1.3.3\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"103.2.0+up1.3.2\",\"app_version\":\"1.3.2\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"103.1.0+up1.3.1\",\"app_version\":\"1.3.1\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"103.0.0+up1.3.0\",\"app_version\":\"1.3.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"102.2.0+up1.2.3\",\"app_version\":\"1.2.3\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"102.1.4+up1.2.2\",\"app_version\":\"1.2.2\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"102.1.1+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"102.1.0+up1.2.1\",\"app_version\":\"1.2.1\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"102.0.2+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"102.0.1+up1.1.5\",\"app_version\":\"1.1.5\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"102.0.0+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator\",\"version\":\"101.0.0+up1.1.5\",\"app_version\":\"1.1.5\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"104.1.0+up1.9.1\",\"app_version\":\"1.9.1\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"104.0.1+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"104.0.0+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"103.4.0+up1.3.4\",\"app_version\":\"1.3.4\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"103.3.0+up1.3.3\",\"app_version\":\"1.3.3\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"103.2.0+up1.3.2\",\"app_version\":\"1.3.2\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"103.1.0+up1.3.1\",\"app_version\":\"1.3.1\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"103.0.0+up1.3.0\",\"app_version\":\"1.3.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"102.2.0+up1.2.3\",\"app_version\":\"1.2.3\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"102.1.4+up1.2.2\",\"app_version\":\"1.2.2\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"102.1.1+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"102.1.0+up1.2.1\",\"app_version\":\"1.2.1\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"102.0.2+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"102.0.1+up1.1.5\",\"app_version\":\"1.1.5\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"102.0.0+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-chart/rancher-eks-operator-crd\",\"version\":\"101.0.0+up1.1.5\",\"app_version\":\"1.1.5\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"104.1.0+up1.9.1\",\"app_version\":\"1.9.1\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"104.0.1+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"104.0.0+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"103.4.0+up1.3.4\",\"app_version\":\"1.3.4\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"103.3.0+up1.3.3\",\"app_version\":\"1.3.3\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"103.2.0+up1.3.2\",\"app_version\":\"1.3.2\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"103.1.0+up1.3.1\",\"app_version\":\"1.3.1\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"103.0.0+up1.3.0\",\"app_version\":\"1.3.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"102.2.0+up1.2.3\",\"app_version\":\"1.2.3\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"102.1.4+up1.2.2\",\"app_version\":\"1.2.2\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"102.1.1+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"102.1.0+up1.2.1\",\"app_version\":\"1.2.1\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"102.0.2+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"102.0.1+up1.1.5\",\"app_version\":\"1.1.5\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"102.0.0+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator\",\"version\":\"101.0.0+up1.1.5\",\"app_version\":\"1.1.5\",\"description\":\"A Helm chart for provisioning EKS clusters\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"104.1.0+up1.9.1\",\"app_version\":\"1.9.1\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"104.0.1+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"104.0.0+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"103.4.0+up1.3.4\",\"app_version\":\"1.3.4\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"103.3.0+up1.3.3\",\"app_version\":\"1.3.3\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"103.2.0+up1.3.2\",\"app_version\":\"1.3.2\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"103.1.0+up1.3.1\",\"app_version\":\"1.3.1\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"103.0.0+up1.3.0\",\"app_version\":\"1.3.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"102.2.0+up1.2.3\",\"app_version\":\"1.2.3\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"102.1.4+up1.2.2\",\"app_version\":\"1.2.2\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"102.1.1+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"102.1.0+up1.2.1\",\"app_version\":\"1.2.1\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"102.0.2+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"102.0.1+up1.1.5\",\"app_version\":\"1.1.5\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"102.0.0+up1.2.0\",\"app_version\":\"1.2.0\",\"description\":\"EKS Operator CustomResourceDefinitions\"},{\"name\":\"rancher-charts/rancher-eks-operator-crd\",\"version\":\"101.0.0+up1.1.5\",\"app_version\":\"1.1.5\",\"description\":\"EKS Operator CustomResourceDefinitions\"}]\n" STEP: downgrading the chart version @ 09/18/24 19:41:53.146 "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 13:44:49.689564112 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 13:44:57.760335064 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"}]\n" "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 13:44:49.689564112 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 13:44:57.760335064 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"}]\n" "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 19:41:54.674361293 +0530 IST\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.1.0+up1.9.1\",\"app_version\":\"1.9.1\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 19:42:06.391984011 +0530 IST\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.1.0+up1.9.1\",\"app_version\":\"1.9.1\"}]\n" STEP: making a change to the cluster (scaling the node up) to validate functionality after chart downgrade @ 09/18/24 19:42:07.232 time="2024-09-18T19:42:07+05:30" level=info msg="ClusterID: c-9rxks, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" time="2024-09-18T19:42:08+05:30" level=info msg="ClusterID: c-9rxks, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T19:42:08+05:30" level=info msg="ClusterID: c-9rxks, Message: [operation error EKS: UpdateNodegroupConfig, https response error StatusCode: 409, RequestID: 2714f2f3-96ae-4fab-ae6f-17c1c33744de, ResourceInUseException: Nodegroup cannot be updated as it is currently not in Active State], Error: true, State: updating, Transitioning: false" time="2024-09-18T19:42:13+05:30" level=info msg="ClusterID: c-9rxks, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T19:42:41+05:30" level=info msg="ClusterID: c-9rxks, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in EKSStatus.UpstreamSpec ..." STEP: uninstalling the operator chart @ 09/18/24 19:46:32.23 "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 19:41:54.674361293 +0530 IST\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.1.0+up1.9.1\",\"app_version\":\"1.9.1\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 19:42:06.391984011 +0530 IST\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.1.0+up1.9.1\",\"app_version\":\"1.9.1\"}]\n" STEP: making a change(adding a nodepool) to the cluster to re-install the operator and validating it is re-installed to the latest/upgraded version @ 09/18/24 19:46:33.629 STEP: ensuring that the chart is re-installed to the latest/upgraded version @ 09/18/24 19:46:33.731 "level"=0 "msg"="[]\n" "level"=0 "msg"="[]\n" "level"=0 "msg"="[{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 14:16:39.266439459 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"}]\n" time="2024-09-18T19:46:44+05:30" level=info msg="ClusterID: c-9rxks, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" time="2024-09-18T19:47:58+05:30" level=info msg="ClusterID: c-9rxks, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T19:47:58+05:30" level=info msg="ClusterID: c-9rxks, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T19:49:25+05:30" level=info msg="ClusterID: c-9rxks, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" Deleting all nodegroups ... Running command: eksctl get nodegroup --region=ap-south-1 --cluster=auto-eks-hp-ci-vfgqt -ojson | jq -r .[].Name Running command: eksctl [delete nodegroup --region=ap-south-1 --name=auto-ng-ttgsq --cluster=auto-eks-hp-ci-vfgqt --disable-eviction --wait] Running command: eksctl [delete nodegroup --region=ap-south-1 --name=ranchernodes --cluster=auto-eks-hp-ci-vfgqt --disable-eviction --wait] Deleting EKS cluster ... Running command: eksctl [delete cluster --region=ap-south-1 --name=auto-eks-hp-ci-vfgqt] Deleted EKS cluster: auto-eks-hp-ci-vfgqt STEP: Installing Rancher back to its original version 2.9.0 @ 09/18/24 19:55:03.625 "level"=0 "msg"="Deploying rancher: [upgrade --install rancher rancher-latest/rancher --namespace cattle-system --create-namespace --set hostname=ec2-65-0-56-221.ap-south-1.compute.amazonaws.com --set bootstrapPassword=adminadmin --set replicas=1 --set global.cattle.psp.enabled=false --wait --version 2.9.0]" STEP: Uninstalling the existing operator charts @ 09/18/24 19:57:43.882 "level"=0 "msg"="[{\"name\":\"rancher-eks-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 14:16:46.257661252 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"},{\"name\":\"rancher-eks-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 14:16:39.266439459 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-eks-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"}]\n" • [2035.049 seconds] ------------------------------ Ran 1 of 2 Specs in 2035.049 seconds SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 1 Skipped PASS | FOCUSED ```
AKS - https://github.com/valaparthvi/hosted-providers-e2e/actions/runs/10919677423
GKE Provisioning( https://github.com/valaparthvi/hosted-providers-e2e/actions/runs/10920755266/job/30311366298 )
``` Run make e2e-k8s-chart-support-provisioning-tests-upgrade go install -mod=mod github.com/onsi/ginkgo/v2/ginkgo go install -mod=mod github.com/onsi/gomega go mod tidy ginkgo -v -r --timeout=3h --keep-going --randomize-all --randomize-suites --focus "K8sChartSupportUpgradeProvisioning" ./hosted/gke/k8s_chart_support/upgrade Running Suite: K8sChartSupportUpgrade Suite - /home/pvala/actions-runner/_work/hosted-providers-e2e/hosted-providers-e2e/hosted/gke/k8s_chart_support/upgrade ============================================================================================================================================================= Random Seed: 1726657489 - will randomize all specs Will run 1 of 2 specs ------------------------------ K8sChartSupportUpgradeProvisioning should successfully test k8s chart support provisioning in an upgrade scenario /home/pvala/actions-runner/_work/hosted-providers-e2e/hosted-providers-e2e/hosted/gke/k8s_chart_support/upgrade/k8s_chart_support_provisioning_upgrade_test.go:36 STEP: Adding the necessary chart repos @ 09/18/24 11:04:58.92 STEP: Installing Rancher Manager v2.9.0 @ 09/18/24 11:04:59.279 "level"=0 "msg"="Deploying rancher: [upgrade --install rancher rancher-latest/rancher --namespace cattle-system --create-namespace --set hostname=34.131.238.15.sslip.io --set bootstrapPassword=*** --set replicas=1 --set global.cattle.psp.enabled=false --wait --version 2.9.0]" "level"=0 "msg"="Using Common SynchronizedBeforeSuite ..." "level"=0 "msg"="Using Common BeforeSuite ..." "level"=0 "msg"="Using GKE version 1.29.8-gke.1211000 for cluster auto-gke-hp-ci-needg" time="2024-09-18T11:15:45Z" level=info msg="Cluster status is active!" "level"=0 "msg"="Testing K8s 1.30 chart support for provisioning on Rancher upgraded from 2.9.0 to devel/2.9" STEP: checking cluster name is same @ 09/18/24 11:15:45.32 STEP: checking service account token secret @ 09/18/24 11:15:45.32 STEP: checking all management nodes are ready @ 09/18/24 11:15:45.336 time="2024-09-18T11:15:45Z" level=info msg="serviceAccountTokenSecret in this cluster is: cluster-serviceaccounttoken-9cq25" STEP: checking all pods are ready @ 09/18/24 11:15:45.353 time="2024-09-18T11:15:45Z" level=info msg="All nodes in the cluster are in an active state!" STEP: checking the chart version @ 09/18/24 11:16:10.45 "level"=0 "msg"="[***\"name\":\"rancher-gke-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 11:07:24.831060644 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-104.0.1+up1.9.0\",\"app_version\":\"1.9.0\"***,***\"name\":\"rancher-gke-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 11:07:17.802790787 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-crd-104.0.1+up1.9.0\",\"app_version\":\"1.9.0\"***]\n" "level"=0 "msg"="Original chart version: 104.0.1+up1.9.0" STEP: upgrading rancher @ 09/18/24 11:16:10.503 "level"=0 "msg"="Deploying rancher: [upgrade --install rancher rancher-latest/rancher --namespace cattle-system --create-namespace --set hostname=34.131.238.15.sslip.io --set bootstrapPassword=*** --set replicas=1 --set global.cattle.psp.enabled=false --wait --devel --set rancherImageTag=v2.9-head]" STEP: ensuring operator pods are also up @ 09/18/24 11:18:55.042 STEP: regenerating the token and initiating a new rancher client @ 09/18/24 11:18:55.291 STEP: making sure the local cluster is ready @ 09/18/24 11:18:56.145 STEP: checking all management nodes are ready @ 09/18/24 11:18:56.145 STEP: checking all pods are ready @ 09/18/24 11:18:56.158 time="2024-09-18T11:18:56Z" level=info msg="All nodes in the cluster are in an active state!" STEP: checking the chart version and validating it is > the old version @ 09/18/24 11:19:01.237 "level"=0 "msg"="[***\"name\":\"rancher-gke-operator\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 11:18:17.20454302 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"***,***\"name\":\"rancher-gke-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 11:18:12.197996425 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"***]\n" "level"=0 "msg"="[***\"name\":\"rancher-gke-operator\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 11:18:17.20454302 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"***,***\"name\":\"rancher-gke-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 11:18:12.197996425 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"***]\n" "level"=0 "msg"="Upgraded chart version: 104.2.0+up1.9.2" STEP: fetching a list of available k8s versions and ensuring v1.30 is present in the list and upgrading the cluster to it @ 09/18/24 11:19:01.37 "level"=0 "msg"="Kubernetes version for cluster auto-gke-hp-ci-needg will be upgraded to 1.30.4-gke.1348000" time="2024-09-18T11:19:01Z" level=info msg="ClusterID: c-ctmr6, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" time="2024-09-18T11:19:01Z" level=info msg="ClusterID: c-ctmr6, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T11:19:01Z" level=info msg="ClusterID: c-ctmr6, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T11:20:33Z" level=info msg="ClusterID: c-ctmr6, Message: [cluster health check failed: Failed to communicate with API server during namespace check: Get \"https://34.126.217.166/api/v1/namespaces/kube-system?timeout=45s\": context deadline exceeded], Error: true, State: error, Transitioning: false" time="2024-09-18T11:21:38Z" level=info msg="ClusterID: c-ctmr6, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T11:30:37Z" level=info msg="ClusterID: c-ctmr6, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" "level"=0 "msg"="Waiting for k8s upgrade to appear in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the nodepool upgrade to appear in GKEStatus.UpstreamSpec ..." STEP: fetching a value to downgrade to @ 09/18/24 11:30:38.046 "level"=0 "msg"="[***\"name\":\"rancher-gke-operator\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 11:18:17.20454302 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"***,***\"name\":\"rancher-gke-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 11:18:12.197996425 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"***]\n" "level"=0 "msg"="[***\"name\":\"rancher-charts/rancher-gke-operator\",\"version\":\"104.2.0+up1.9.2\",\"app_version\":\"1.9.2\",\"description\":\"A Helm chart for provisioning GKE clusters\"***,***\"name\":\"rancher-charts/rancher-gke-operator\",\"version\":\"104.1.0+up1.9.1\",\"app_version\":\"1.9.1\",\"description\":\"A Helm chart for provisioning GKE clusters\"***,***\"name\":\"rancher-charts/rancher-gke-operator\",\"version\":\"104.0.1+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"A Helm chart for provisioning GKE clusters\"***,***\"name\":\"rancher-charts/rancher-gke-operator\",\"version\":\"104.0.0+up1.9.0\",\"app_version\":\"1.9.0\",\"description\":\"A Helm chart for provisioning GKE clusters\"***,***\"name\":\"rancher-charts/rancher-gke-operator\",\"version\":\"103.4.0+up1.2.4\",\"app_version\":\"1.2.4\",\"description\":\"A Helm chart for provisioning GKE clusters\"***,***\"name\":\"rancher-charts/rancher-gke-operator\",\"version\":\"103.3.0+up1.2.3\",\"app_version\":\"1.2.3\",\"description\ STEP: downgrading the chart version @ 09/18/24 11:30:38.33 "level"=0 "msg"="[***\"name\":\"rancher-gke-operator\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 11:18:17.20454302 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"***,***\"name\":\"rancher-gke-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 11:18:12.197996425 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"***]\n" "level"=0 "msg"="[***\"name\":\"rancher-gke-operator\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 11:18:17.20454302 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"***,***\"name\":\"rancher-gke-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"2\",\"updated\":\"2024-09-18 11:18:12.197996425 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"***]\n" "level"=0 "msg"="[***\"name\":\"rancher-gke-operator\",\"namespace\":\"cattle-system\",\"revision\":\"3\",\"updated\":\"2024-09-18 11:30:39.021702771 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-104.1.0+up1.9.1\",\"app_version\":\"1.9.1\"***,***\"name\":\"rancher-gke-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"3\",\"updated\":\"2024-09-18 11:30:48.602756747 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-crd-104.1.0+up1.9.1\",\"app_version\":\"1.9.1\"***]\n" STEP: making a change to the cluster (scaling nodepool up) to validate functionality after chart downgrade @ 09/18/24 11:30:48.869 time="2024-09-18T11:30:48Z" level=info msg="ClusterID: c-ctmr6, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" time="2024-09-18T11:30:50Z" level=info msg="ClusterID: c-ctmr6, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T11:30:50Z" level=info msg="ClusterID: c-ctmr6, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T11:31:54Z" level=info msg="ClusterID: c-ctmr6, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" "level"=0 "msg"="Waiting for the node count change to appear in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the node count change to appear in GKEStatus.UpstreamSpec ..." STEP: uninstalling the operator chart @ 09/18/24 11:32:44.589 "level"=0 "msg"="[***\"name\":\"rancher-gke-operator\",\"namespace\":\"cattle-system\",\"revision\":\"3\",\"updated\":\"2024-09-18 11:30:39.021702771 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-104.1.0+up1.9.1\",\"app_version\":\"1.9.1\"***,***\"name\":\"rancher-gke-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"3\",\"updated\":\"2024-09-18 11:30:48.602756747 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-crd-104.1.0+up1.9.1\",\"app_version\":\"1.9.1\"***]\n" STEP: making a change(adding a nodepool) to the cluster to re-install the operator and validating it is re-installed to the latest/upgraded version @ 09/18/24 11:32:44.883 STEP: ensuring that the chart is re-installed to the latest/upgraded version @ 09/18/24 11:32:44.918 "level"=0 "msg"="[]\n" "level"=0 "msg"="[]\n" "level"=0 "msg"="[]\n" "level"=0 "msg"="[***\"name\":\"rancher-gke-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 11:32:56.170017527 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"***]\n" time="2024-09-18T11:33:00Z" level=info msg="ClusterID: c-ctmr6, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T11:33:00Z" level=info msg="ClusterID: c-ctmr6, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T11:34:30Z" level=info msg="ClusterID: c-ctmr6, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." "level"=0 "msg"="Waiting for the total nodepool count to increase in GKEStatus.UpstreamSpec ..." STEP: Installing Rancher back to its original version 2.9.0 @ 09/18/24 11:37:50.391 "level"=0 "msg"="Deploying rancher: [upgrade --install rancher rancher-latest/rancher --namespace cattle-system --create-namespace --set hostname=34.131.238.15.sslip.io --set bootstrapPassword=*** --set replicas=1 --set global.cattle.psp.enabled=false --wait --version 2.9.0]" STEP: Uninstalling the existing operator charts @ 09/18/24 11:40:15.52 "level"=0 "msg"="[***\"name\":\"rancher-gke-operator\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 11:32:56.170017527 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"***,***\"name\":\"rancher-gke-operator-crd\",\"namespace\":\"cattle-system\",\"revision\":\"1\",\"updated\":\"2024-09-18 11:33:03.226371459 +0000 UTC\",\"status\":\"deployed\",\"chart\":\"rancher-gke-operator-crd-104.2.0+up1.9.2\",\"app_version\":\"1.9.2\"***]\n" Qase ID 62 created for run ID 1000 on project HP • [2116.857 seconds] ------------------------------ S Ran 1 of 2 Specs in 2119.148 seconds SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 1 Skipped PASS Ginkgo ran 1 suite in 35m28.552053189s Test Suite Passed ```
GKE Import
``` Running Suite: K8sChartSupportUpgrade Suite - /home/pvala/go/src/github.com/rancher/hosted-providers-e2e/hosted/gke/k8s_chart_support/upgrade ============================================================================================================================================= Random Seed: 1726654057 Will run 1 of 2 specs SCreating GKE cluster ... Running command: gcloud [container clusters create auto-gke-hp-ci-ylefg --project container-project-qa --zone asia-south2-c --cluster-version 1.29.8-gke.1211000 --labels owner=hosted-providers-qa-ci-pvala,testfilenumber=line40_k8s_chart_support_import_upgrade_test --network default --release-channel None --machine-type n2-standard-2 --disk-size 100 --num-nodes 1 --no-enable-master-authorized-networks] Created GKE cluster: auto-gke-hp-ci-ylefg time="2024-09-18T15:47:38+05:30" level=info msg="Cluster status is active!" time="2024-09-18T15:47:38+05:30" level=info msg="serviceAccountTokenSecret in this cluster is: cluster-serviceaccounttoken-xgdwb" time="2024-09-18T15:47:38+05:30" level=info msg="All nodes in the cluster are in an active state!" time="2024-09-18T15:50:32+05:30" level=info msg="All nodes in the cluster are in an active state!" time="2024-09-18T15:50:38+05:30" level=info msg="ClusterID: c-2g8wf, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" time="2024-09-18T15:50:39+05:30" level=info msg="ClusterID: c-2g8wf, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T15:50:39+05:30" level=info msg="ClusterID: c-2g8wf, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T15:52:24+05:30" level=info msg="ClusterID: c-2g8wf, Message: [cluster health check failed: Failed to communicate with API server during namespace check: Get \"https://34.131.233.134/api/v1/namespaces/kube-system?timeout=45s\": context deadline exceeded], Error: true, State: error, Transitioning: false" time="2024-09-18T15:53:15+05:30" level=info msg="ClusterID: c-2g8wf, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T16:01:45+05:30" level=info msg="ClusterID: c-2g8wf, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" time="2024-09-18T16:01:59+05:30" level=info msg="ClusterID: c-2g8wf, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" time="2024-09-18T16:02:01+05:30" level=info msg="ClusterID: c-2g8wf, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T16:02:01+05:30" level=info msg="ClusterID: c-2g8wf, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T16:03:04+05:30" level=info msg="ClusterID: c-2g8wf, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" time="2024-09-18T16:06:57+05:30" level=info msg="ClusterID: c-2g8wf, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T16:06:57+05:30" level=info msg="ClusterID: c-2g8wf, Message: [], Error: false, State: updating, Transitioning: true" time="2024-09-18T16:08:29+05:30" level=info msg="ClusterID: c-2g8wf, Message: [Resource is Ready], Error: false, State: active, Transitioning: false" Deleting GKE cluster ... Running command: gcloud [container clusters delete auto-gke-hp-ci-ylefg --zone asia-south2-c --quiet --project container-project-qa --async] Deleted GKE cluster: auto-gke-hp-ci-ylefg • Ran 1 of 2 Specs in 2206.091 seconds SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 1 Skipped ```
What does this PR do?
This PR fixes k8s chart support upgrade tests ~and adds GH action workflow file~. These tests were originally written to test k8s upgrade with a new chart release, but since it involves upgrading Rancher, it can be used to test Rancher upgrade and ensure things work even after the upgrade.
Which issue(s) this PR fixes (optional, in fixes #(, fixes #, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Checklist:
Special notes for your reviewer: