search

fiaas / fiaas-deploy-daemon

fiaas-deploy-daemon is the core component of the FIAAS platform
https://fiaas.github.io/
Apache License 2.0
55 stars 31 forks source link

Include failure events from Deployment/Replicaset/Pod in case of timeout being 'ready' #56

Open gregjones opened 4 years ago

gregjones commented 4 years ago

We've seen users having problems debugging deploy errors in a couple of cases:

My idea is to capture these events when fiaas decides the deployment is failed due to a timeout, and attach them to the ApplicationStatus object, in a similar way to what is done for logs, so they can be displayed in the mast UI.

mortenlj commented 4 years ago

While I like the idea, I'm wondering if this should be outside the responsibilities of fiaas-deploy-daemon. It already has too many responsibilities in my mind, I'm not sure it's a good idea to give it more.

On the other hand, the ApplicationStatus object is an object that can be said to be owned by FDD, so in some ways you could say that FDD should be the one adding additional information.

There are also some open questions:

Could it instead be some other component that watches for these types of errors and attaches k8s Events to the ApplicationStatus object?

gregjones commented 4 years ago

For how long should FDD be watching for liveness/readiness check failures?

I think the existing behaviour of ReadyCheck is fine for this, it's just an extra thing for it to do just before it sets the status as failed.

Should these events be added as a log-line, or do we need a new field on the ApplicationStatus object for them?

I think as a separate property gives more flexility for how we display them to users, but extra log-lines could work.

Could it instead be some other component that watches for these types of errors and attaches k8s Events to the ApplicationStatus object?

It's an option, though I'm not sure using ApplicationStatus would make so much sense if it's another component, it seems like mixing ownership could become confusing. But something that watches ApplicationStatus objects and stores the events somewhere else that can be referenced somehow from a UI could work