search

apache / openwhisk-deploy-kube

The Apache OpenWhisk Kubernetes Deployment repository supports deploying the Apache OpenWhisk system on Kubernetes and OpenShift clusters.
https://openwhisk.apache.org/
Apache License 2.0
302 stars 232 forks source link

controller pod with status "ImageInspectError" #634

Closed sgallizia closed 3 years ago

sgallizia commented 4 years ago

I followed this guide https://github.com/apache/openwhisk-deploy-kube/blob/master/docs/k8s-docker-for-windows.md to install Openwhisk on Docker Desktop, but I can't complete the installation. The controller pod is stuck in status "ImageInspectError":

PS C:\WINDOWS\system32> kubectl get all -n openwhisk -o wide
NAME                                       READY   STATUS              RESTARTS   AGE   IP          NODE             NOMINATED NODE   READINESS GATES
pod/owdev-alarmprovider-6f584689bf-tlb49   0/1     Init:0/1            0          45m   10.1.0.94   docker-desktop   <none>           <none>
pod/owdev-apigateway-6d864b779d-5t6gx      1/1     Running             0          45m   10.1.0.93   docker-desktop   <none>           <none>
pod/owdev-controller-0                     0/1     ImageInspectError   0          45m   10.1.0.92   docker-desktop   <none>           <none>
pod/owdev-couchdb-647d5945ff-q9c22         1/1     Running             0          45m   10.1.0.88   docker-desktop   <none>           <none>
pod/owdev-gen-certs-whv84                  0/1     Completed           0          45m   10.1.0.56   docker-desktop   <none>           <none>
pod/owdev-init-couchdb-2t6wp               0/1     Completed           0          45m   10.1.0.58   docker-desktop   <none>           <none>
pod/owdev-install-packages-4jphf           0/1     Init:0/1            0          45m   10.1.0.95   docker-desktop   <none>           <none>
pod/owdev-invoker-0                        0/1     Init:0/1            0          45m   10.1.0.97   docker-desktop   <none>           <none>
pod/owdev-kafka-0                          1/1     Running             0          45m   10.1.0.90   docker-desktop   <none>           <none>
pod/owdev-kafkaprovider-795b455d7-pn5g9    0/1     Init:0/1            0          45m   10.1.0.91   docker-desktop   <none>           <none>
pod/owdev-nginx-85f9fdc44-tglds            0/1     Init:0/1            0          45m   10.1.0.83   docker-desktop   <none>           <none>
pod/owdev-redis-9bf746976-v5wmm            1/1     Running             0          45m   10.1.0.96   docker-desktop   <none>           <none>
pod/owdev-tests-package-checker            0/1     Error               0          66m   10.1.0.44   docker-desktop   <none>           <none>
pod/owdev-wskadmin                         1/1     Running             0          45m   10.1.0.82   docker-desktop   <none>           <none>
pod/owdev-zookeeper-0                      1/1     Running             0          45m   10.1.0.89   docker-desktop   <none>           <none>

NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE   SELECTOR
service/owdev-apigateway   ClusterIP   10.108.222.120   <none>        8080/TCP,9000/TCP            45m   name=owdev-apigateway
service/owdev-controller   ClusterIP   10.101.21.44     <none>        8080/TCP                     45m   name=owdev-controller
service/owdev-couchdb      ClusterIP   10.105.21.23     <none>        5984/TCP                     45m   name=owdev-couchdb
service/owdev-kafka        ClusterIP   None             <none>        9092/TCP                     45m   name=owdev-kafka
service/owdev-nginx        NodePort    10.110.129.115   <none>        80:31690/TCP,443:31001/TCP   45m   name=owdev-nginx
service/owdev-redis        ClusterIP   10.100.150.138   <none>        6379/TCP                     45m   name=owdev-redis
service/owdev-zookeeper    ClusterIP   None             <none>        2181/TCP,2888/TCP,3888/TCP   45m   name=owdev-zookeeper

NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS      IMAGES                          SELECTOR
deployment.apps/owdev-alarmprovider   0/1     1            0           45m   alarmprovider   openwhisk/alarmprovider:2.2.0   name=owdev-alarmprovider
deployment.apps/owdev-apigateway      1/1     1            1           45m   apigateway      openwhisk/apigateway:1.0.0      name=owdev-apigateway
deployment.apps/owdev-couchdb         1/1     1            1           45m   couchdb         apache/couchdb:2.3              name=owdev-couchdb
deployment.apps/owdev-kafkaprovider   0/1     1            0           45m   kafkaprovider   openwhisk/kafkaprovider:2.1.0   name=owdev-kafkaprovider
deployment.apps/owdev-nginx           0/1     1            0           45m   nginx           nginx:1.17                      name=owdev-nginx
deployment.apps/owdev-redis           1/1     1            1           45m   redis           redis:4.0                       name=owdev-redis

NAME                                             DESIRED   CURRENT   READY   AGE   CONTAINERS      IMAGES                          SELECTOR
replicaset.apps/owdev-alarmprovider-6f584689bf   1         1         0       45m   alarmprovider   openwhisk/alarmprovider:2.2.0   name=owdev-alarmprovider,pod-template-hash=6f584689bf
replicaset.apps/owdev-apigateway-6d864b779d      1         1         1       45m   apigateway      openwhisk/apigateway:1.0.0      name=owdev-apigateway,pod-template-hash=6d864b779d
replicaset.apps/owdev-couchdb-647d5945ff         1         1         1       45m   couchdb         apache/couchdb:2.3              name=owdev-couchdb,pod-template-hash=647d5945ff
replicaset.apps/owdev-kafkaprovider-795b455d7    1         1         0       45m   kafkaprovider   openwhisk/kafkaprovider:2.1.0   name=owdev-kafkaprovider,pod-template-hash=795b455d7
replicaset.apps/owdev-nginx-85f9fdc44            1         1         0       45m   nginx           nginx:1.17                      name=owdev-nginx,pod-template-hash=85f9fdc44
replicaset.apps/owdev-redis-9bf746976            1         1         1       45m   redis           redis:4.0                       name=owdev-redis,pod-template-hash=9bf746976

NAME                                READY   AGE   CONTAINERS   IMAGES
statefulset.apps/owdev-controller   0/1     45m   controller   openwhisk/controller:71b7d56
statefulset.apps/owdev-invoker      0/1     45m   invoker      openwhisk/invoker:71b7d56
statefulset.apps/owdev-kafka        1/1     45m   kafka        wurstmeister/kafka:2.12-2.3.1
statefulset.apps/owdev-zookeeper    1/1     45m   zookeeper    zookeeper:3.4

NAME                               COMPLETIONS   DURATION   AGE   CONTAINERS         IMAGES                       SELECTOR
job.batch/owdev-gen-certs          1/1           9s         45m   gen-certs          openwhisk/ow-utils:71b7d56   controller-uid=aeab52e2-5663-4a0f-98bd-51e76d5e87ce
job.batch/owdev-init-couchdb       1/1           66s        45m   init-couchdb       openwhisk/ow-utils:71b7d56   controller-uid=dc9d358e-95f9-45ae-8741-1e16857a94e1
job.batch/owdev-install-packages   0/1           45m        45m   install-packages   openwhisk/ow-utils:71b7d56   controller-uid=564f793c-8d7f-4020-931b-19f8181e6be2

Docker Desktop 2.3.0.4 Engine 19.03.12 Kubertenes 1.16.5

dgrove-oss commented 4 years ago

try deleting the pod and see if the problem recurs. Some googling suggests that this is often a transient failure.

sgallizia commented 4 years ago

In the meantime I installed an older version of Docker Desktop (2.1.0.1) and now the situation seems normal:

PS C:\openwhisk\openwhisk-deploy-kube> kubectl get all -n openwhisk -o wide
NAME                                         READY   STATUS      RESTARTS   AGE     IP          NODE             NOMINATED NODE   READINESS GATES
pod/owdev-alarmprovider-f7c4454cb-hxrw5      1/1     Running     0          7m10s   10.1.0.22   docker-desktop   <none>           <none>
pod/owdev-apigateway-648fb5664f-vcx92        1/1     Running     0          7m10s   10.1.0.15   docker-desktop   <none>           <none>
pod/owdev-controller-0                       1/1     Running     0          7m9s    10.1.0.20   docker-desktop   <none>           <none>
pod/owdev-couchdb-5858d8cf6b-m22hz           1/1     Running     0          7m10s   10.1.0.24   docker-desktop   <none>           <none>
pod/owdev-gen-certs-7sgxj                    0/1     Completed   0          7m10s   10.1.0.16   docker-desktop   <none>           <none>
pod/owdev-init-couchdb-9kfz9                 0/1     Completed   0          7m10s   10.1.0.18   docker-desktop   <none>           <none>
pod/owdev-install-packages-zq8gq             1/1     Running     0          7m10s   10.1.0.17   docker-desktop   <none>           <none>
pod/owdev-invoker-0                          1/1     Running     0          7m9s    10.1.0.19   docker-desktop   <none>           <none>
pod/owdev-kafka-0                            1/1     Running     0          7m9s    10.1.0.23   docker-desktop   <none>           <none>
pod/owdev-kafkaprovider-6486d7d98b-qhw9k     1/1     Running     0          7m10s   10.1.0.14   docker-desktop   <none>           <none>
pod/owdev-nginx-6cb965b769-trbdg             1/1     Running     0          7m10s   10.1.0.26   docker-desktop   <none>           <none>
pod/owdev-redis-97b88b94f-svbmj              1/1     Running     0          7m10s   10.1.0.21   docker-desktop   <none>           <none>
pod/owdev-wskadmin                           1/1     Running     0          7m10s   10.1.0.13   docker-desktop   <none>           <none>
pod/owdev-zookeeper-0                        1/1     Running     0          7m10s   10.1.0.25   docker-desktop   <none>           <none>
pod/wskowdev-invoker-00-7-prewarm-nodejs10   1/1     Running     0          50s     10.1.0.33   docker-desktop   <none>           <none>
pod/wskowdev-invoker-00-8-prewarm-nodejs10   1/1     Running     0          49s     10.1.0.34   docker-desktop   <none>           <none>

NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE     SELECTOR
service/owdev-apigateway   ClusterIP   10.96.163.39     <none>        8080/TCP,9000/TCP            7m10s   name=owdev-apigateway
service/owdev-controller   ClusterIP   10.104.102.89    <none>        8080/TCP                     7m10s   name=owdev-controller
service/owdev-couchdb      ClusterIP   10.109.146.110   <none>        5984/TCP                     7m10s   name=owdev-couchdb
service/owdev-kafka        ClusterIP   None             <none>        9092/TCP                     7m10s   name=owdev-kafka
service/owdev-nginx        NodePort    10.98.187.4      <none>        80:32555/TCP,443:31001/TCP   7m10s   name=owdev-nginx
service/owdev-redis        ClusterIP   10.108.116.47    <none>        6379/TCP                     7m10s   name=owdev-redis
service/owdev-zookeeper    ClusterIP   None             <none>        2181/TCP,2888/TCP,3888/TCP   7m10s   name=owdev-zookeeper

NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS      IMAGES                          SELECTOR
deployment.apps/owdev-alarmprovider   1/1     1            1           7m10s   alarmprovider   openwhisk/alarmprovider:2.2.0   name=owdev-alarmprovider
deployment.apps/owdev-apigateway      1/1     1            1           7m10s   apigateway      openwhisk/apigateway:1.0.0      name=owdev-apigateway
deployment.apps/owdev-couchdb         1/1     1            1           7m10s   couchdb         apache/couchdb:2.3              name=owdev-couchdb
deployment.apps/owdev-kafkaprovider   1/1     1            1           7m10s   kafkaprovider   openwhisk/kafkaprovider:2.1.0   name=owdev-kafkaprovider
deployment.apps/owdev-nginx           1/1     1            1           7m10s   nginx           nginx:1.17                      name=owdev-nginx
deployment.apps/owdev-redis           1/1     1            1           7m10s   redis           redis:4.0                       name=owdev-redis

NAME                                             DESIRED   CURRENT   READY   AGE     CONTAINERS      IMAGES                          SELECTOR
replicaset.apps/owdev-alarmprovider-f7c4454cb    1         1         1       7m10s   alarmprovider   openwhisk/alarmprovider:2.2.0   name=owdev-alarmprovider,pod-template-hash=f7c4454cb
replicaset.apps/owdev-apigateway-648fb5664f      1         1         1       7m10s   apigateway      openwhisk/apigateway:1.0.0      name=owdev-apigateway,pod-template-hash=648fb5664f
replicaset.apps/owdev-couchdb-5858d8cf6b         1         1         1       7m10s   couchdb         apache/couchdb:2.3              name=owdev-couchdb,pod-template-hash=5858d8cf6b
replicaset.apps/owdev-kafkaprovider-6486d7d98b   1         1         1       7m10s   kafkaprovider   openwhisk/kafkaprovider:2.1.0   name=owdev-kafkaprovider,pod-template-hash=6486d7d98b
replicaset.apps/owdev-nginx-6cb965b769           1         1         1       7m10s   nginx           nginx:1.17                      name=owdev-nginx,pod-template-hash=6cb965b769
replicaset.apps/owdev-redis-97b88b94f            1         1         1       7m10s   redis           redis:4.0                       name=owdev-redis,pod-template-hash=97b88b94f

NAME                                READY   AGE     CONTAINERS   IMAGES
statefulset.apps/owdev-controller   1/1     7m10s   controller   openwhisk/controller:71b7d56
statefulset.apps/owdev-invoker      1/1     7m10s   invoker      openwhisk/invoker:71b7d56
statefulset.apps/owdev-kafka        1/1     7m10s   kafka        wurstmeister/kafka:2.12-2.3.1
statefulset.apps/owdev-zookeeper    1/1     7m10s   zookeeper    zookeeper:3.4

NAME                               COMPLETIONS   DURATION   AGE     CONTAINERS         IMAGES                       SELECTOR
job.batch/owdev-gen-certs          1/1           59s        7m10s   gen-certs          openwhisk/ow-utils:71b7d56   controller-uid=25089de6-f2ac-11ea-b6d9-00155d014f03
job.batch/owdev-init-couchdb       1/1           2m12s      7m10s   init-couchdb       openwhisk/ow-utils:71b7d56   controller-uid=2508c806-f2ac-11ea-b6d9-00155d014f03
job.batch/owdev-install-packages   0/1           7m10s      7m10s   install-packages   openwhisk/ow-utils:71b7d56   controller-uid=2508edbb-f2ac-11ea-b6d9-00155d014f03

So, it seems the problem happens only with newer version of Docker Desktop.

dgrove-oss commented 3 years ago

Closing as the user was able to get it to work and it doesn't appear actionable for us.