Expand the fields considered in local caching

[waciumawanjohi]

As a cartographer ops user When I have a template that creates any kind of object I want cartographer to minimizes the updates of that object

Problem:

Cartographer does local caching to ensure that Supply Chain does not update an object continually (instead only does so when there would be a change to the object). Pipeline similarly only creates an object when the new object would differ from the old. When doing this, Cartographer considers changes that would have been applied by a mutating webhook on submission. In this comparison, Cartographer considers only the spec field. Cartographer should handle objects that have other input fields. (e.g. ConfigMaps have a data field).

Given a Supply Chain with a *Template that creates a K8s object without a spec field
When the resource version of the object is observed
Then the version has not blown up to some large value

Given a Pipeline with a RunTemplate that creates a K8s object without a spec field
When the resource is observed
Then only one copy of the object is found

Questions

Should we consider metadata, like labels? Are there examples of controllers that behave differently depending on the labels on an object? (Trivially, yes. Cartographer!)

Notes

If this story is not targeted for Beta, then there should be a Beta story written for documenting the current behavior (described in the problem above).

[cirocosta]

waciumawanjohi [...] Are there examples of controllers that behave differently depending on the labels on an object? (Trivially, yes. Cartographer!)

knative is one example where one might customize its controller to behave differently whenever it spots a particular annotation/label

e.g., to opt-out of the default domain schema that it uses:

kind: ConfigMap
data:
  domainTemplate: |-
    {{if index .Annotations "custom-hostname" -}}{{- index .Annotations "custom-        hostname" -}}
    {{else -}}{{- .Name}}.{{.Namespace -}}.{{.Domain}}{{end -}}

so that you could create a knative service like

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: rng
  annotations:
    custom-hostname: rng.cartographer.sh
spec:
  template:
    spec:
      containers:
        - image: ko://github.com/vmware-tanzu/cartographer/cmd/rng

that would serve our "rng application" under rng.cartographer.sh

(although yes, knative itself is also very opinionated about its immutable labels)

[emmjohnson]

In pre-IPM we talked about caching everything besides Status. @cirocosta brought up metadata.resourceversion - so we probably want to exclude that.. maybe there are others??

[cirocosta]

interesting enough, now that we've introduced the emission of events for the action of applying a stamped object to the cluter we can quite clearly see how objects that don't make use of spec end up with no use of the cache and thus getting patched on every reconciliation

e.g., consider the following supply chain:

apiVersion: carto.run/v1alpha1
kind: ClusterSupplyChain
metadata:
  name: supplychain
spec:
  selector:
    foo: bar

  resources:
    - name: first
      templateRef:
        kind: ClusterTemplate
        name: template
---
apiVersion: carto.run/v1alpha1
kind: ClusterTemplate
metadata:
  name: template
spec:
  template:
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: $(workload.metadata.name)$-templated
    data:
      foo: bar

submitting a workload that matches its selector, we can see that on every reconciliation of the workload, a patch event gets surfaced:

Events:
  Type    Reason                Age                From      Message
  ----    ------                ----               ----      -------
  Normal  StampedObjectApplied  65s                Workload  Created object [configmaps/workload-templated]
  Normal  StampedObjectApplied  57s (x3 over 63s)  Workload  Patched object [configmaps/workload-templated]
              '--- should not even patched at all as we didn't change anything
                     in either the template or the workload