search

rancher / rancher

Complete container management platform
http://rancher.com
Apache License 2.0
23.46k stars 2.98k forks source link

Rancher does not update helm packages correctly #23541

Open TheAifam5 opened 5 years ago

TheAifam5 commented 5 years ago

What kind of request is this (question/bug/enhancement/feature request): bug

Steps to reproduce (least amount of steps as possible):

  1. Install RKE with v1.15.3-rancher1-1
  2. Install rancher 2.2.9
  3. Install docker-registry 1.7.0
  4. Upgrade rancher to 2.3.1
  5. Upgrade RKE to v1.16.2-rancher1-1
  6. Upgrade docker-registry to 1.8.3

Result:

Failed to install app docker-registry. Error: UPGRADE FAILED: failed decoding reader into objects: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1"

Other details that may be helpful: docker-registry:1.8.3 has added the support for K8S 1.16.2.

Environment information

Cluster information

Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.2", GitCommit:"f6278300bebbb750328ac16ee6dd3aa7d3549568", GitTreeState:"archive", BuildDate:"2019-08-29T18:43:18Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.2", GitCommit:"c97fe5036ef3df2967d086711e6c0c405941e14b", GitTreeState:"clean", BuildDate:"2019-10-15T19:09:08Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}

Server: Docker Engine - Community Engine: Version: 19.03.2 API version: 1.40 (minimum version 1.12) Go version: go1.12.8 Git commit: 6a30dfca03 Built: Thu Aug 29 05:32:56 2019 OS/Arch: linux/amd64 Experimental: false containerd: Version: v1.2.6 GitCommit: 894b81a4b802e4eb2a91d1ce216b8817763c29fb runc: Version: 1.0.0-rc8 GitCommit: 425e105d5a03fabd737a126ad93d62a9eeede87f docker-init: Version: 0.18.0 GitCommit: fec3683

jiaqiluo commented 5 years ago

@TheAifam5 This is a known issue that the current Docker-registry template in Rancher's built-in catalog library (template version 1.4.3 and 1.6.1) does not work on k8s 1.16 cluster, and there will be a new version of the template in the coming Rancher v2.3.2 release.

In your case, you said you installed docker-registry 1.7.0 then upgraded to 1.8.3, are those the version of the template or the docker images? Did you use the template from Rancher's built-in catalog library or others?

TheAifam5 commented 5 years ago

It’s the template version. I think thats official one from rancher repo, because the helm repo has already been upgraded for k8s 1.6 version as I said.

jiaqiluo commented 5 years ago

@TheAifam5 Thanks for the feedback. I found the template from the office Helm repo (https://kubernetes-charts.storage.googleapis.com) which I need to enable in Rancher -> Apps -> Manage Catalogs

In a v2.3.1 setup, I tried the following steps quickly and did not see the issue you reported

- Run Rancher:v2.3.1 single install
- Add a cluster with k8s `v1.15.5-rancher1-2` using EC2 nodes
- Enable the Helm repo in Rancher -> Apps -> Manage Catalogs
- deploy the app `docker-registry` template version 1.7.0
- wait to see the app is active 
- upgrade the cluster's k8s to `v1.16.2-rancher1-1` 
- upgrade the app `docker-registry` to template version 1.8.3
- the upgrade is successful

Your steps sound like you are deploying the app to the same cluster as Rancher is running, is it right? I will try your steps later to see if I can reproduce it.

TheAifam5 commented 5 years ago

Yes, your right. for now I downgraded k8s to 1.15 and clicked on „upgrade“ again and all works for now.

I will wait for the chart update for k8s 1.16. Its was my production environment so I can’t disrupt the services now.

aleksey005 commented 5 years ago

Rancher 2.3.2, Kubernetes Version: v1.16.2

Apps not upgrade from Library:

Docker Registry template 1.8.1 Failed to install app docker-registry. Error: UPGRADE FAILED: failed decoding reader into objects: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1"

Drone template 2.0.7 Failed to install app drone. Error: UPGRADE FAILED: failed decoding reader into objects: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1"

MySQL template 1.3.1 Failed to install app mysql. Error: UPGRADE FAILED: failed decoding reader into objects: unable to recognize "": no matches for kind "Deployment" in version "apps/v1beta1"

TheAifam5 commented 4 years ago

Is there a way to edit the template manually in Rancher? The nfs-client-provisioner got updated, but upgrading it does not do anything / or how can I reinstall the helm package without breaking everything?

rene-demonsters commented 4 years ago

Same issue as @aleksey005 but on Rancher v2.3.3 and k8s 1.16.2 and trying to update longhorn.

aleksey005 commented 4 years ago

@rene-demonsters Apparently the application library is abandoned, like the official helm. Recently, more and more companies are leaving the official helm with the status of a deprecated and making their repositories, for myself I concluded that it is better not to use the rancher library or helm because these applications will not be updated if there are official charts.

aleksey005 commented 4 years ago

I found that it is possible to enable affected api, which would probably solve problems with broken charts, but it is not clear how to do this in the installation of the rancher without destroying its work. Maybe someone has a case how to do it?

/etc/kubernetes/manifests/kube-apiserver.yaml

spec:
    ...
    - --runtime-config=apps/v1beta1=true,apps/v1beta2=true,extensions/v1beta1/daemonsets=true,extensions/v1beta1/deployments=true,extensions/v1beta1/replicasets=true,extensions/v1beta1/networkpolicies=true,extensions/v1beta1/podsecuritypolicies=true

and kubectl -n kube-system delete pod kube-apiserver-whatever kube-controller-manager-whatever