Add defrag CronJob

[kvaps]

Kubernetes recomends using etcd-defrag to run defragmentation of the cluster.

Since this is expensive operation this tool allows to specify specific rule, eg:

dbQuotaUsage > 0.8 || dbSize - dbSizeInUse > 200*1024*1024

https://github.com/ahrtr/etcd-defrag/blob/main/doc/etcd-defrag-cronjob.yaml

This is slightly diferent from kamaji-etcd implementation https://github.com/clastix/kamaji-etcd/blob/433989a4badeb8edc0925cfc671fb0cc66448e66/charts/kamaji-etcd/templates/etcd_cronjob_defrag.yaml#L11

In Cozystack we currnelty added it into Helm chart: https://github.com/aenix-io/cozystack/pull/137/files#diff-0dcf3889fea1c88eb22a0d84208697a4ed71c8cb111d1d29ec714a71529d5822

[kvaps]

@Kirill-Garbar commented:

Let's discuss whether it's a cron job or not, just that. Personally, I don't like cron jobs even though they are simple

[kvaps]

Probably depends on https://github.com/aenix-io/etcd-operator/pull/204

[kvaps]

---
apiVersion: etcd.aenix.io/v1alpha1
kind: EtcdCluster
metadata:
  name: test
  namespace: test-tls-auth-etcd-cluster
spec:
  options:
  defrag:
    schedule: "*/15 * * * *"
    rule: # TODO: check the common Kubernetes syntax
     operator: Or|And
     quotaMoreThan: 80%
     dbFreeSizeLessThan: 200MB
  storage:
    volumeClaimTemplate:
      spec:
        storageClassName: gp3
        accessModes: [ "ReadWriteOnce" ]
        resources:
          requests:
            storage: 10Gi
  security:
    enableAuth: true
    tls:
      peerTrustedCASecret: ca-peer-secret
      peerSecret: peer-secret
      serverTrustedCASecret: ca-server-secret
      serverSecret: server-secret
      clientTrustedCASecret: ca-client-secret
      clientSecret: client-secret
 status:
   conditions:
   - lastProbeTime: null
     lastTransitionTime: "2024-03-06T18:39:45Z"
     status: "True"
     type: DefragChecked

[kvaps]

@lllamnyp also sugested to use go templates:

rule: {{ and (gt .dbAvailableSize 200*1024*1024) (gt .dbInUsePercent 80) }}

[kvaps]

defragJobTerms:
- matchExpressions:
  - key: dbSpaceFree
    operator: Lt
    values: 200Mi
  - key: dbQuotaUsage
    operator: Gt
    values: 20%