Closed mhulscher closed 7 years ago
bacongobbler
commented
7 years ago I know the helm team is still working on compatibility with v1.5.0 so installing through conventional means is not even a possibility at this time. I'm assuming you installed using helm-classic?
Bregor
commented
7 years ago $ helm version
Client: &version.Version{SemVer:"v2.0.2", GitCommit:"afcab3bb4f45478772a853e0d98306fda0a7af34", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.0.2", GitCommit:"afcab3bb4f45478772a853e0d98306fda0a7af34", GitTreeState:"clean"}I tried that and hit https://github.com/kubernetes/helm/issues/1591#issuecomment-266842591.
Bregor
commented
7 years ago In my particular case tiller-deploy is already installed since k8s-1.4.
Problem is not in helm install ... but in git push deis master
Bregor
commented
7 years ago Also operations like deis config:set, deis create ... and so on fails constantly with
deis[controller]: ERROR (list pods): 'NoneType' object is not iterable
mhulscher
commented
7 years ago Just like @Bregor I installed Deis using Helm 2.0.0 when the K8s-cluster was still 1.4.6. This morning I tried upgrading K8s to 1.5.0. This is an on-premise sandbox cluster bootstrapped with kubeadm.
mboersma
commented
7 years ago It's possible that this is due to API access being denied in k8s v1.5.0, as described here: https://marc.ttias.be/kubernetes-announce/2016-12/msg00004.php
tl:dr we should re-test this with v1.5.1.
bacongobbler
commented
7 years ago I have re-tested this with v1.5.1 and helm v2.1.0. I can install workflow again, but it still seems like deis ps still fails with the following:
><> deis ps -a go
Error: Unknown Error (503): {"detail":"(list pods): 'NoneType' object is not iterable"}controller logs:
10.246.74.20 "GET /v2/apps/go/pods/?limit=100 HTTP/1.1" 503 59 "Deis Client dev-eb21ef2"
ERROR [go]: (list pods): 'NoneType' object is not iterable
ERROR:root:'NoneType' object is not iterable
Traceback (most recent call last):
File "/app/api/models/app.py", line 742, in list_pods
for p in pods:
TypeError: 'NoneType' object is not iterable
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/rest_framework/views.py", line 474, in dispatch
response = handler(request, *args, **kwargs)
File "/app/api/views.py", line 305, in list
pods = self.get_app().list_pods(*args, **kwargs)
File "/app/api/models/app.py", line 779, in list_pods
raise ServiceUnavailable(err) from e
api.exceptions.ServiceUnavailable: (list pods): 'NoneType' object is not iterableGoing to sleep but will retry testing in the morning.
bacongobbler
commented
7 years ago So it looks like they changed the response in 1.5.0 when fetching pods when none exist:
2016-12-14T18:07:51+00:00 deis[controller]: INFO {'items': None, 'kind': 'PodList', 'metadata': {'resourceVersion': '5350', 'selfLink': '/api/v1/namespaces/go/pods'}, 'apiVersion': 'v1'}notice that the items field is None. In v1.4.7 that would be an empty list, so we will need to account for that.
bacongobbler
commented
7 years ago this has now been merged into master. Up to @mboersma whether we want to cut a v2.9.1 with this fix or wait until v2.10 for k8s 1.5 compatibility. :)
Bregor
commented
7 years ago v2.9.1, please
Creating an application w/ latest Deis and Kubernetes 1.5.0.
deis-controller logs (collected w/ stern)
Rolling back the control-plane to 1.4.7 fixed the issue. I am available if more information is required.