Temp directory in memory

[ngoeddel-openi]

Is your feature request related to a problem ?

With a small enough refresh interval for the source and kustomize controller they can create a lot of disk I/O on a system. In our case it is just too much, the kustomize pod constantly uses 200% CPU and creates a lot of writes to the underlying file system. Unfortunately there is no possibility to change the volume mount for /tmp. It is just a simply emptyDir volume. https://github.com/fluxcd-community/helm-charts/blob/main/charts/flux2/templates/kustomize-controller.yaml#L134

Describe the solution you'd like.

My solution would be to add additional parameters for emptyDir or somehow using .Values.kustomizeController.volumes to override the temp volume.

The first idea would be the easiest one I guess:

      ...
      volumes:
      - name: temp
        emptyDir:
          {{ .Values.kustomizeController.tempOptions | toYaml | nindent 10 }}
      ...

Then there should be a default for that in values.yaml:

kustomizeController:
  tempOptions: {}

Which can be overridden like this:

kustomizeController:
  tempOptions:
    medium: Memory
    sizeLimit: 128Mi

Describe alternatives you've considered.

NONE

Additional context.

No response

[stefanprodan]

What’s the reason for using anything under 10m for Flux Kustomization interval? That interval is for undoing kubectl edits. See the recommended setting here: https://fluxcd.io/flux/components/kustomize/kustomizations/#recommended-settings

[ngoeddel-openi]

@stefanprodan Thanks for your answer. Good point indeed. Since this is a development environment I am talking about there are a lot of changes all the time, and because of that we thought it would be a good idea to make the interval on every Flux resource very small. Apparently we did not read the documentation well enough to find out that the Kustomization interval is only for checking drifts and manual edits.

So just one final question for you from my side: Am I right with the assumption that a small refresh interval for GitRepo is fine and it only pulls new changes if the commit hash of the given branch has changed? So basically there is no downside regarding too much disk I/O on the source controller side with git repositories? And if there was a change in a git repository it automatically triggers the Kustomization controller anyway?

If all these questions can be answered with "Yes", then I guess we can close this ticket.

[stefanprodan]

Am I right with the assumption that a small refresh interval for GitRepo is fine and it only pulls new changes if the commit hash of the given branch has changed?

Yes, what we've implemented in source-controller is the equivalent of git-ls-remote. We write to disk, only if the remote HEAD commit is different from what we have previously pulled in the cluster.

And if there was a change in a git repository it automatically triggers the Kustomization controller anyway?

Yes, all Flux controllers are event-based, if a source revision changes, an event is issued and no matter the interval, the changes get reconciled immediately.

[ngoeddel-openi]

Thanks a lot! I guess this knowledge will make this ticket obsolete. Closing it right now.