Open yanqiangmiffy opened 1 year ago
kubectl get pods --all-namespaces 其中有些pod状态为Pending
kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE auth dex-559dbcd758-tk7kz 1/1 Running 0 3h47m cert-manager cert-manager-7cc4df4cd6-tc2rs 1/1 Running 1 3h47m cert-manager cert-manager-cainjector-8d785bf4c-6nfz9 1/1 Running 1 3h47m cert-manager cert-manager-webhook-87648988f-rgb9g 1/1 Running 0 3h47m default volume-test 1/1 Running 0 32m istio-system authservice-0 0/1 Pending 0 3h47m istio-system cluster-local-gateway-5449f87d9b-46g47 1/1 Running 0 3h47m istio-system istio-ingressgateway-77b9d69b74-w4d4h 1/1 Running 0 3h47m istio-system istiod-67fcb675b5-77rnh 1/1 Running 0 3h47m knative-eventing eventing-controller-5c847567d9-lngqt 1/1 Running 0 74m knative-eventing eventing-webhook-7f7fbb679f-9m97f 1/1 Running 0 130m knative-serving activator-76f4874c56-xx2fw 2/2 Running 15 130m knative-serving autoscaler-df6fb584d-g2kwt 2/2 Running 0 63m knative-serving controller-9b64997bb-dl4wk 2/2 Running 0 63m knative-serving domain-mapping-c475b7674-krbwh 2/2 Running 0 63m knative-serving domainmapping-webhook-599b77575f-b92vl 2/2 Running 0 130m knative-serving net-istio-controller-554557d85f-8ldqc 2/2 Running 0 130m knative-serving net-istio-webhook-5dd85b4967-sz2cn 2/2 Running 0 130m knative-serving webhook-6b494b646c-8w8pf 2/2 Running 0 63m kube-system calico-kube-controllers-5bcd7db644-gdsj9 1/1 Running 0 4h5m kube-system calico-node-bmfsr 1/1 Running 3 4h5m kube-system coredns-59d64cd4d4-6z8pz 1/1 Running 0 4h41m kube-system coredns-59d64cd4d4-ks8f2 1/1 Running 0 4h41m kube-system etcd-server-szry1agd.novalocal 1/1 Running 0 4h41m kube-system kube-apiserver-server-szry1agd.novalocal 1/1 Running 0 4h41m kube-system kube-controller-manager-server-szry1agd.novalocal 1/1 Running 11 4h41m kube-system kube-proxy-qgdwl 1/1 Running 0 4h41m kube-system kube-scheduler-server-szry1agd.novalocal 1/1 Running 10 4h41m kubeflow admission-webhook-deployment-c77d48bbb-2nbdt 1/1 Running 0 3h30m kubeflow cache-server-6647b76c64-m2rpv 2/2 Running 3 43m kubeflow centraldashboard-5864f74d99-9w9hk 2/2 Running 0 3h30m kubeflow jupyter-web-app-deployment-844cf8dd7c-mncll 1/1 Running 0 3h30m kubeflow katib-controller-6848d4dd9f-j9wbl 1/1 Running 0 3h30m kubeflow katib-db-manager-665954948-5c9xd 0/1 CrashLoopBackOff 43 3h30m kubeflow katib-mysql-5bf95ddfcc-5mw64 0/1 Pending 0 3h30m kubeflow katib-ui-56ccff658f-6ljnh 1/1 Running 0 3h30m kubeflow kserve-controller-manager-0 2/2 Running 0 3h47m kubeflow kserve-models-web-app-544c6b55bb-znttv 2/2 Running 1 3h30m kubeflow kubeflow-pipelines-profile-controller-94bf4544-m6ss8 1/1 Running 0 3h30m kubeflow metacontroller-0 1/1 Running 0 3h47m kubeflow metadata-envoy-deployment-868cf55d45-twrwm 1/1 Running 0 43m kubeflow metadata-grpc-deployment-f8d68f687-b2pvq 1/2 CrashLoopBackOff 34 3h30m kubeflow metadata-writer-d6498d6b4-9j7cr 2/2 Running 27 3h30m kubeflow minio-5b65df66c9-gclvx 0/2 Pending 0 3h30m kubeflow ml-pipeline-67cf96c9bf-98vws 1/2 CrashLoopBackOff 41 3h30m kubeflow ml-pipeline-persistenceagent-5854f86f8b-9b7rm 2/2 Running 0 3h30m kubeflow ml-pipeline-scheduledworkflow-5dddbf664f-gbtsc 2/2 Running 0 3h30m kubeflow ml-pipeline-ui-6bdfc6dbcd-5gxl6 2/2 Running 2 3h30m kubeflow ml-pipeline-viewer-crd-84f7566c48-z6ntx 2/2 Running 1 43m kubeflow ml-pipeline-visualizationserver-54fccfcb5f-6hw7d 2/2 Running 1 52m kubeflow mysql-5c7f79f986-nhh2k 0/2 Pending 0 3h30m kubeflow notebook-controller-deployment-9954c4fd9-6hrg4 2/2 Running 1 3h30m kubeflow tensorboard-controller-deployment-844d9585fc-s4rjj 3/3 Running 1 3h30m kubeflow tensorboards-web-app-deployment-7578c885f7-jq7hz 1/1 Running 0 3h30m kubeflow training-operator-6c9f6fd894-8fdcx 1/1 Running 0 3h30m kubeflow volumes-web-app-deployment-7bc5754bd4-47fft 1/1 Running 0 3h30m kubeflow workflow-controller-6b9b6c5b46-dthnc 2/2 Running 5 3h30m local-path-storage local-path-provisioner-7b6c8448c7-g7w6g 1/1 Running 0 153m
然后查看其中描述 kubectl describe pod authservice-0 -n istio-system
kubectl describe pod authservice-0 -n istio-system
Name: authservice-0 Namespace: istio-system Priority: 0 Service Account: default Node: <none> Labels: app=authservice controller-revision-hash=authservice-6db8b4db64 statefulset.kubernetes.io/pod-name=authservice-0 Annotations: sidecar.istio.io/inject: false Status: Pending IP: IPs: <none> Controlled By: StatefulSet/authservice Containers: authservice: Image: gcr.io/arrikto/kubeflow/oidc-authservice:28c59ef Port: 8080/TCP Host Port: 0/TCP Readiness: http-get http://:8081/ delay=0s timeout=1s period=10s #success=1 #failure=3 Environment Variables from: oidc-authservice-client Secret Optional: false oidc-authservice-parameters ConfigMap Optional: false Environment: <none> Mounts: /var/lib/authservice from data (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-hsh6m (ro) Conditions: Type Status PodScheduled False Volumes: data: Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace) ClaimName: authservice-pvc ReadOnly: false kube-api-access-hsh6m: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: <nil> DownwardAPI: true QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedScheduling 46m (x1 over 47m) default-scheduler 0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims. Warning FailedScheduling 31m default-scheduler 0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims. Warning FailedScheduling 55s (x29 over 30m) default-scheduler 0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims.
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-6b18608d-7141-4e9e-b294-c7780e86e747 128Mi RWO Delete Bound default/local-path-pvc local-path 35m
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE local-path-pvc Bound pvc-6b18608d-7141-4e9e-b294-c7780e86e747 128Mi RWO local-path 36m
- kubectl get pod
NAME READY STATUS RESTARTS AGE volume-test 1/1 Running 0 36m
可以试下删除pvc 和pod,看重新创建有没有问题
kubectl get pods --all-namespaces
其中有些pod状态为Pending然后查看其中描述
kubectl describe pod authservice-0 -n istio-system
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE local-path-pvc Bound pvc-6b18608d-7141-4e9e-b294-c7780e86e747 128Mi RWO local-path 36m
NAME READY STATUS RESTARTS AGE volume-test 1/1 Running 0 36m