search

Kupenstack / kupenstack

Kubernetes-Native Openstack
https://kupenstack.io
Apache License 2.0
19 stars 12 forks source link

deployment appears stuck for 8+ hours, on Ubuntu 18.04 and minikube., also tried kind. #27

Open outbackdingo opened 2 years ago

outbackdingo commented 2 years ago

deployment appears stuck for 8+ hours NAME READY STATUS RESTARTS AGE horizon-7f4f6d9fcb-rg9ks 0/1 Init:0/1 0 8h horizon-db-init--1-mp7dm 0/1 Init:0/1 0 8h horizon-db-sync--1-vlbq6 0/1 Init:0/1 0 8h ingress-d5d8b9bdc-hh2wq 0/1 CrashLoopBackOff 155 (5s ago) 8h ingress-error-pages-6d54c96946-bnlbl 1/1 Running 0 8h keystone-api-85b965f486-47hzx 0/1 Init:0/2 0 8h keystone-db-init--1-bf96g 0/1 Init:0/1 0 8h keystone-fernet-rotate-27300960--1-q75ns 0/1 Init:0/1 0 129m kupenstack-controller-678c66b58c-sxlwv 1/1 Running 0 8h mariadb-ingress-56bbb59f78-mlncq 0/1 CrashLoopBackOff 183 (3m50s ago) 8h mariadb-ingress-error-pages-77c55c9cd7-fvr6k 1/1 Running 0 8h mariadb-server-0 1/1 Running 0 8h memcached-memcached-9978cc66c-vvr7s 1/1 Running 0 8h neutron-db-init--1-9wfm5 0/1 Init:0/1 0 8h neutron-dhcp-agent-default-x7f5w 0/1 Init:0/2 0 8h neutron-l3-agent-default-pj7ch 0/1 Init:0/2 0 8h neutron-lb-agent-default-6qx2k 0/1 Init:0/3 0 8h neutron-metadata-agent-default-44tbh 0/1 Init:0/2 0 8h neutron-netns-cleanup-cron-default-4lkf8 1/1 Running 0 8h neutron-server-656ccbf745-vts2f 0/1 Init:0/1 0 8h nova-api-metadata-779bbbfd6b-qhdr4 0/1 Init:0/2 0 8h nova-api-osapi-79bffdb64c-xbk8x 0/1 Init:0/1 0 8h nova-bootstrap--1-vzqtb 0/1 Init:0/2 0 8h nova-cell-setup--1-ljtcn 0/1 Init:0/2 0 8h nova-conductor-955f5c4b-vm4kh 0/1 Init:0/1 0 8h nova-consoleauth-545bd5cd69-kzzzf 0/1 Init:0/1 0 8h nova-db-init--1-f7jms 0/3 Init:0/1 0 8h nova-novncproxy-7f94c99b6f-znz2z 0/1 Init:0/3 0 8h nova-scheduler-6fff8d5b86-stdsp 0/1 Init:0/1 0 8h rabbitmq-rabbitmq-0 1/1 Running 0 8h

Alt-Shivam commented 2 years ago

Hey @outbackdingo Can you please provide me with your environment info So that I can rectify this issue.

outbackdingo commented 2 years ago

ubuntu 18.04 kubectl deployed kubernetes, rook-ceph

outbackdingo commented 2 years ago

i also just tried this with kind as per your document and it failed kubectl get pods -n kupenstack NAME READY STATUS RESTARTS AGE glance-api-7649d4d869-b8qnk 0/1 Init:0/2 0 8m21s glance-db-init-bnp9j 0/1 Init:0/1 0 8m21s horizon-57b49b87d5-cghlx 0/1 Init:0/1 0 8m49s horizon-db-init-2j5j5 0/1 Init:0/1 0 8m49s horizon-db-sync-lv79c 0/1 Init:0/1 0 8m49s ingress-d5d8b9bdc-2f9sl 1/1 Running 0 9m20s ingress-error-pages-6d54c96946-vrphv 1/1 Running 0 9m20s keystone-api-795996c9fd-7mzxq 0/1 Init:0/2 0 8m49s keystone-credential-setup-pg77j 0/1 Completed 0 8m49s keystone-db-init-dj6w5 0/1 Init:0/1 0 6m12s kupenstack-controller-678c66b58c-drksl 1/1 Running 0 9m53s libvirt-libvirt-default-g8stp 0/1 Init:0/1 0 9m19s mariadb-ingress-556ff7f4ff-nstvl 0/1 CrashLoopBackOff 3 75s mariadb-ingress-error-pages-7d9d8c67fd-htkcw 1/1 Running 0 9m20s mariadb-server-0 1/1 Running 0 9m20s memcached-memcached-9978cc66c-m9lbw 1/1 Running 0 9m21s neutron-db-init-sb4lq 0/1 Init:0/1 0 8m49s neutron-dhcp-agent-default-k5957 0/1 Init:0/2 0 8m50s neutron-l3-agent-default-2h86s 0/1 Init:0/2 0 8m50s neutron-lb-agent-default-vjdgf 0/1 Init:0/3 0 8m50s neutron-metadata-agent-default-6fddm 0/1 Init:0/2 0 8m50s neutron-netns-cleanup-cron-default-jwj2q 1/1 Running 0 8m50s neutron-server-6fb5c6975f-mbs88 0/1 Init:0/1 0 8m50s nova-api-metadata-5667c5769f-76nq6 0/1 Init:0/2 0 8m49s nova-api-osapi-6d6ffcc55b-ht7n8 0/1 Init:0/1 0 8m49s nova-bootstrap-fcfnd 0/1 Init:0/2 0 8m49s nova-cell-setup-wnxdn 0/1 Init:0/3 0 8m49s nova-compute-default-szhf4 0/1 Init:0/3 0 8m49s nova-conductor-5d46cd5899-299zp 0/1 Init:0/1 0 8m49s nova-consoleauth-5fdb5f769d-clbz7 0/1 Init:0/1 0 8m49s nova-db-init-2wjzt 0/3 Init:0/1 0 8m49s nova-novncproxy-654794b96d-mnb62 0/1 Init:0/3 0 8m49s nova-scheduler-5b48bf85f-xmxlw 0/1 Init:0/1 0 8m49s placement-api-747d998dcf-t6rr8 0/1 Init:0/1 0 8m50s placement-db-init-hjl9s 0/1 Init:0/1 0 8m50s placement-db-sync-nnz52 0/1 Init:0/1 0 8m50s rabbitmq-cluster-wait-kkf87 0/1 Init:0/2 0 9m22s rabbitmq-rabbitmq-0 0/1 Running 1 9m22s

outbackdingo commented 2 years ago

kubectl describe pod mariadb-ingress-556ff7f4ff-nstvl -n kupenstack Name: mariadb-ingress-556ff7f4ff-nstvl Namespace: kupenstack Priority: 0 Node: kind-control-plane/172.19.0.2 Start Time: Sat, 05 Feb 2022 18:45:04 +0700 Labels: app.kubernetes.io/component=ingress app.kubernetes.io/instance=mariadb-ingress app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=mariadb app.kubernetes.io/version=v10.2.31 application=mariadb component=ingress pod-template-hash=556ff7f4ff release_group=mariadb Annotations: configmap-bin-hash: bbdd6113686adcf6cd70ba5888f5eb864ff6bada7478807bc9631f9a8175db42 configmap-etc-hash: 319cef174bf71ffbbd4111b78c70f178ba4d7cc55365e688a764ed3cde40a519 openstackhelm.openstack.org/release_uuid: Status: Running IP: 10.244.0.36 IPs: IP: 10.244.0.36 Controlled By: ReplicaSet/mariadb-ingress-556ff7f4ff Init Containers: init: Container ID: containerd://62b4618b37ec06265538ae79453486fa17cca1cc5f849c2f5bf28d35cc2cb054 Image: quay.io/airshipit/kubernetes-entrypoint:v1.0.0 Image ID: sha256:c092d0dada614fdae3920939c5a9683b2758288f23c2e3b425128653857d7520 Port: Host Port: Command: kubernetes-entrypoint State: Terminated Reason: Completed Exit Code: 0 Started: Sat, 05 Feb 2022 18:45:06 +0700 Finished: Sat, 05 Feb 2022 18:45:08 +0700 Ready: True Restart Count: 0 Environment: POD_NAME: mariadb-ingress-556ff7f4ff-nstvl (v1:metadata.name) NAMESPACE: kupenstack (v1:metadata.namespace) INTERFACE_NAME: eth0 PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/ DEPENDENCY_SERVICE: kupenstack:mariadb-ingress-error-pages DEPENDENCY_DAEMONSET:
DEPENDENCY_CONTAINER:
DEPENDENCY_POD_JSON:
DEPENDENCY_CUSTOM_RESOURCE:
Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-tpmk4 (ro) Containers: ingress: Container ID: containerd://bccd4e5cc6396571cefa0a79a10181e6714a7e7e60233f1f22a9723aec6fe3b2 Image: k8s.gcr.io/ingress-nginx/controller:v0.42.0 Image ID: k8s.gcr.io/ingress-nginx/controller@sha256:f7187418c647af4a0039938b0ab36c2322ac3662d16be69f9cc178bfd25f7eee Port: Host Port: Command: /tmp/mariadb-ingress-controller.sh start State: Waiting Reason: CrashLoopBackOff Last State: Terminated Reason: Error Exit Code: 255 Started: Sat, 05 Feb 2022 18:46:52 +0700 Finished: Sat, 05 Feb 2022 18:46:52 +0700 Ready: False Restart Count: 4 Liveness: http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=3 Readiness: tcp-socket :3306 delay=0s timeout=1s period=10s #success=1 #failure=3 Environment: POD_NAME: mariadb-ingress-556ff7f4ff-nstvl (v1:metadata.name) POD_NAMESPACE: kupenstack (v1:metadata.namespace) RELEASE_NAME: mariadb INGRESS_CLASS: mariadb-mariadb-ingress ERROR_PAGE_SERVICE: mariadb-ingress-error-pages Mounts: /etc/nginx/template/nginx.tmpl from mariadb-ingress-etc (ro,path="nginx.tmpl") /tmp from pod-tmp (rw) /tmp/mariadb-ingress-controller.sh from mariadb-bin (ro,path="mariadb-ingress-controller.sh") /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-tpmk4 (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: pod-tmp: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium:
SizeLimit: mariadb-bin: Type: ConfigMap (a volume populated by a ConfigMap) Name: mariadb-bin Optional: false mariadb-ingress-etc: Type: ConfigMap (a volume populated by a ConfigMap) Name: mariadb-ingress-etc Optional: false kube-api-access-tpmk4: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: DownwardAPI: true QoS Class: BestEffort Node-Selectors: openstack-control-plane=enabled Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message

Normal Scheduled 2m30s default-scheduler Successfully assigned kupenstack/mariadb-ingress-556ff7f4ff-nstvl to kind-control-plane Normal Pulled 2m29s kubelet Container image "quay.io/airshipit/kubernetes-entrypoint:v1.0.0" already present on machine Normal Created 2m29s kubelet Created container init Normal Started 2m28s kubelet Started container init Normal Pulled 90s (x4 over 2m26s) kubelet Container image "k8s.gcr.io/ingress-nginx/controller:v0.42.0" already present on machine Normal Created 90s (x4 over 2m26s) kubelet Created container ingress Normal Started 90s (x4 over 2m26s) kubelet Started container ingress Warning Unhealthy 90s (x2 over 90s) kubelet Readiness probe failed: dial tcp 10.244.0.36:3306: connect: connection refused Warning BackOff 89s (x8 over 2m24s) kubelet Back-off restarting failed container