Suppress caching of Secrets and ConfigMaps

squaremo commented 1 year ago

Fixes #23.

As explained in https://github.com/kubernetes-sigs/controller-runtime/pull/1249, the controller-runtime client will tend to use LIST and WATCH to cache resources requested by the controller. This has two downsides:

the mechanism may require broad RBAC access (Secret and ConfigMap in every namespace) even if the actual secrets and configmaps referenced are few, or restricted to few namespaces;
the cache uses a lot of memory that it doesn't really need to

This change fixes those problems, with the trade-off that all Secret and ConfigMap requests use a round-trip to the Kubernetes API server.

This follows https://github.com/fluxcd/source-controller/pull/989. There, a feature flag can be used to restore caching for Secrets and ConfigMaps; I have not included the feature flag, since this controller is much younger and doesn't have an established behaviour.

bigkevmcd commented 1 year ago

Do we still need to watch secrets?

https://github.com/weaveworks/gitopssets-controller/blob/0989d3e8a41da66c9e6811f8ac4d856b226f53ca/controllers/gitopsset_controller.go#L87

squaremo commented 1 year ago

Do we still need to watch secrets?

Good spot :detective: Probably not, but let me verify that.

squaremo commented 1 year ago

Do we still need to watch secrets?

Good spot detective Probably not, but let me verify that.

The controller doesn't watch them (in SetupWithManager), and the controller-runtime doesn't need watch if it's not implicitly caching the resources -> logically, watch is not needed. I've removed it from the kubebuilder annotations.

weaveworks / gitopssets-controller

Suppress caching of Secrets and ConfigMaps #66