Ordering of resource across Kapp deploys

[KauzClay]

Describe the problem/challenge you have For example, let’s say I have an packageinstall kns that deploys knative-serving with contour, and I have another app hello, deployed independently from the pkgi, that just contains a simple knative service. If I delete my kns pkgi, hello will start failing because no knative stuff is on the cluster. When I reapply my pkgi kns, hello will try to get applied right away, before kns has a chance to finish deploying. Most of the time, things sort themselves out, but sometimes hello may be applied to the cluster before Knative is totally ready. Maybe a knative webhook isn't ready, and some defaults don’t get set, or maybe some, but not all, of the CRDs for Knative are applied, so an internal resource doesn't get created as part of the new knative service.

I learned in this thread that it currently is not possible.

Maybe I am wrong about this, but it seems unreasonable to expect a controller to properly reconcile a resource before all of the controller's accompanying resources,svcs,crds,etc are fully ready. I'm not sure how else to avoid this situation.

Describe the solution you'd like I want to be able to make hello wait until kns is totally ready before trying to deploy again.

I can imagine a set of annotations similar to the change group annotations:

kapp.k14s.io/dependent-resource-type: "App" 
kapp.k14s.io/dependent-resource-name: "kns" 
kapp.k14s.io/dependent-resource-namespace: "default" 

Perhaps kapp-controller could use these annotations to check that the described resource is indeed ready before trying to reconcile the app.

Anything else you would like to add: Happy to hear other ideas, including ones that don't involve changing kapp-controller, on how to address this situation 😄

---

Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible" 👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

[joe-kimmel-vmw]

Thanks @KauzClay ! This problem of "I want to use change-group apply ordering and dependent-resource annotations across multiple packages" is part of what we mean by dependency management. Right now @cppforlife is owning that but this is a good specific corner case to consider as input for that track of work.

[cppforlife]

Maybe I am wrong about this, but it seems unreasonable to expect a controller to properly reconcile a resource before all of the controller's accompanying resources,svcs,crds,etc are fully ready. I'm not sure how else to avoid this situation.

controllers can take their sweet time to reconcile things (and that's ok), but of course there are other edge cases like CRD isnt present on the cluster or may be webhook for CRD defaulting isnt there, etc.

I can imagine a set of annotations similar to the change group annotations:

how are you deploying your PackageInstalls? can you just use kapp to deploy them, hence you would be able to use ordering rules? (that's what we have been doing -- sometimes for example we use kapp directly, but sometimes we have an App CR that deploys other things and of course inside you can use existing kapp annotations)

[KauzClay]

how are you deploying your PackageInstalls? can you just use kapp to deploy them, hence you would be able to use ordering rules?

Unfortunately not. The Knative packageInstall would be installed by an operator persona, while the hello app that uses a knative service is created by a developer persona using the cluster