GoogleContainerTools / skaffold

Easy and Repeatable Kubernetes Development
https://skaffold.dev/
Apache License 2.0
14.94k stars 1.62k forks source link

Add final pod status to STATUSCHECK_POD_INITIALIZING output #4668

Closed tstromberg closed 4 years ago

tstromberg commented 4 years ago

As noted by @russwolf:

Status check started
Resource pod/nodejs-guestbook-backend-76c5c46f9c-bbdcn status failed with
Resource pod/nodejs-guestbook-frontend-f748c9d67-cjxs9 status updated to In Progress
Resource pod/nodejs-guestbook-mongodb-6f8b7fd56d-9gskl status completed successfully
Resource deployment/nodejs-guestbook-frontend status updated to In Progress
Resource deployment/nodejs-guestbook-frontend status updated to In Progress
Resource pod/nodejs-guestbook-mongodb-6f8b7fd56d-9gskl status updated to In Progress
Resource deployment/nodejs-guestbook-mongodb status updated to In Progress
Resource deployment/nodejs-guestbook-mongodb status updated to In Progress
Resource deployment/nodejs-guestbook-backend status failed with could not stabilize within 2m0s: context deadline exceeded
Resource deployment/nodejs-guestbook-frontend status failed with could not stabilize within 2m0s: context deadline exceeded
Resource deployment/nodejs-guestbook-mongodb status failed with could not stabilize within 2m0s: context deadline exceeded
Status check failed

DevInit Iteration 0 failed with error code STATUSCHECK_POD_INITIALIZING
exiting dev mode because first deploy failed: 3/3 deployment(s) failed

This error message does not provide enough detail for post-mortem analysis. At a minimum, it needs to show the resulting pod states. Here is my naive suggestion. Instead of:

Resource deployment/nodejs-guestbook-backend status failed with could not stabilize within 2m0s: context deadline exceeded
Resource deployment/nodejs-guestbook-frontend status failed with could not stabilize within 2m0s: context deadline exceeded
Resource deployment/nodejs-guestbook-mongodb status failed with could not stabilize within 2m0s: context deadline exceeded
Status check failed

Show something like the output of kubectl get and kubectl descr

3 of 3 deployments to <kubectl context name> did not complete within the 2m0s timeout: deployment/nodejs-guestbook-backend, deployment/nodejs-guestbook-frontend, deployment/nodejs-guestbook-mongodb

Final state of resources:

deployment/nodejs-guestbook-mongodb: 1/1 ready (18s)
pod/xxx: Running  (12s)
pod/xy: Failed (Evicted): The node was low on resource: memory. 
pod/xy: <.Status> (<.Reason>): <.Message>

FWIW, I'd rather get something in ASAP and worry about the exact UX later. This information is critical for users.

tstromberg commented 4 years ago

/cc @tejal29 @nkubala

tejal29 commented 4 years ago

This might be dupe of #4658 I will look into why events are not fetched when pods are initializing.