aws-samples / eks-cluster-upgrade

Automated Amazon EKS cluster upgrade
MIT No Attribution
154 stars 34 forks source link

Bug: k8s.gcr.io (registry) hardcoded for cluster-autoscaler #113

Closed drmaciej closed 1 year ago

drmaciej commented 1 year ago

Expected Behaviour

No exception thrown, both old and new registry handled.

Current Behaviour

Snippet of the output when running eksupgrade drmaciej-cluster 1.26 ap-southeast-2 --preflight

Cluster Autoscaler exists
cluster-autoscaler pod is running
Error occurred while checking for the cluster autoscaler - Error: list index out of range
Pre flight unsuccessful because of the following errors: ['To upgrade please run the code with --force flag ', 'Error occurred while checking for the cluster autoscaler list index out of range']
Pre-flight check for cluster drmaciej-cluster targeting version: 1.26 failed!

It appears that the exception is thrown in

                version = (
                    i.spec.template.spec.containers[0]
                    .image.split("k8s.gcr.io/autoscaling/cluster-autoscaler:v")[1]
                    .split("-")[0]
                )

because my image is set to registry.k8s.io/autoscaling/cluster-autoscaler:v1.25.0.

k8s.gcr.io is about to be sunset and new images are not published there. For instance, the latest publish CA images are on the new registry already (see https://github.com/kubernetes/autoscaler/releases)

Code snippet

NA

Possible Solution

No response

Steps to Reproduce

Detailed in "Current Behaviour"

Amazon EKS upgrade version

1.25 to 1.26

Python runtime version

3.9

Packaging format used

PyPi

Debugging logs

No response

ricky1-gupta commented 1 year ago

@drmaciej I am also facing the similar issue with the above problem as my image is set to registry.k8s.io/autoscaling/cluster-autoscaler:v1.21.0 .

ricky1-gupta commented 1 year ago

@bryantbiggs I tried without using the pre-flight checks also but its not working. eksupgrade cluster_name cluster_version region ERROR: Error occurred while checking for the cluster autoscaler - Error: list index out of range Pre flight unsuccessful because of the following errors: ['To upgrade please run the code with --force flag ', 'Error occurred while checking for the cluster autoscaler list index out of range'] Pre-flight check for cluster NonProd-Movies-Cluster targeting version: 1.21 failed!

bryantbiggs commented 1 year ago

112 removes the pre-flight checks which is where this issue is coming from https://github.com/aws-samples/eks-cluster-upgrade/pull/112/files#diff-b9aeab33037539343fdb6eb6a9360f1fcfeab0008902d6c840aea7666ef999ea

ricky1-gupta commented 1 year ago

@bryantbiggs Can share with the command for updating the same . As I uninstall and installed it again but the version is still 0.8.3 .

bryantbiggs commented 1 year ago

Once a new release is created, you can upgrade to the next version which will not have this issue

ricky1-gupta commented 1 year ago

@bryantbiggs When will you release it as I need it asap.

bryantbiggs commented 1 year ago

we should be able to get a release out today - cc @mbeacom