Wave currently sets owner references for all secrets/configmaps. That causes a lot of API requests and events on those objects causing further api requests by other controllers. It also causes edge-case bugs such as #79. To prevent that we have to use a finalizer which causes further API requests and delays deletion.
I propose to get rid of all ownerReferences and finalizers. Instead I propose to build up a watch list for all secrets and configmaps during reconcile. We already watch all configmaps and secrets anyway. All we would have to change is that we would look up the namespaced name of the secret/configmap in our watch list and queue a reconcile request for all mapped deployments/statefulsets/daemonsets.
As a migration for current users we would delete all ownerReferences created by wave on startup.
Advantages:
Much less complexity
We can delete a lot of code
Less API calls
Faster reconciles
Less space on disk in etcd
Fixes bugs
Other solutions use a similar approach (but without hashes)
Disadvantages:
New code
Increase in memory in wave to store watch list. This should be neglectable in practice.
I implemented a proof of concept to confirm that this is feasible. It passes our tests and I dont see any other downsides.
Open questions:
Do we want to keep ownerReferences as an option? I personally dont see any real advantages of that so I would delete the code.
How long do we keep the migration code? Until the next major? Forever? Guess that also need documentation.
Do we need to clean up our watcher list when objects change? In my opinion that rarely happens but it would require quite a bit of code and additional memory to store the previous children of an object. Currently, it is only cleaned up when deployments are deleted (or when wave restarts). In my opinion that should be fine in practice.
Are there any concerns or opinions on that topic? If not I would create a PR to change this.
Wave currently sets owner references for all secrets/configmaps. That causes a lot of API requests and events on those objects causing further api requests by other controllers. It also causes edge-case bugs such as #79. To prevent that we have to use a finalizer which causes further API requests and delays deletion.
I propose to get rid of all ownerReferences and finalizers. Instead I propose to build up a watch list for all secrets and configmaps during reconcile. We already watch all configmaps and secrets anyway. All we would have to change is that we would look up the namespaced name of the secret/configmap in our watch list and queue a reconcile request for all mapped deployments/statefulsets/daemonsets.
As a migration for current users we would delete all ownerReferences created by wave on startup.
Advantages:
Disadvantages:
I implemented a proof of concept to confirm that this is feasible. It passes our tests and I dont see any other downsides.
Open questions:
Are there any concerns or opinions on that topic? If not I would create a PR to change this.