Ability to define the coredns and metrics-server pod resources

[fmunteanu]

Environmental Info: K3s Version:

# k3s -v
k3s version v1.30.6+k3s1 (1829eaae)
go version go1.22.8

Node(s) CPU architecture, OS, and Version:

# uname -a
Linux apollo 6.8.0-1014-raspi #16-Ubuntu SMP PREEMPT_DYNAMIC Tue Oct 15 20:54:23 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

Cluster Configuration: 3 servers + 5 agents, HA enabled

Describe the bug: I looked at the server configuration documentation and I could find any relevant setting allowing me to define the coredns and metrics-server resources. I also did a search through closed issues, without finding any relevant information. I see for example the metrics-server pod resources set to:

    Requests:
      cpu:        100m
      memory:     70Mi

Where are these values configured? Ideally, I would like to also set the number of pods to 2 (instead of 1), not sure if this is recommended.

Current server configuration on apollo server:

# cat /etc/rancher/k3s/config.yaml
bind-address: 192.168.4.2
cluster-dns: 10.43.0.10
cluster-domain: cluster.local
disable:
  - local-storage
  - servicelb
  - traefik
disable-cloud-controller: true
disable-kube-proxy: true
disable-network-policy: true
embedded-registry: true
etcd-expose-metrics: true
flannel-backend: none
kube-controller-manager-arg: bind-address=192.168.4.2
kube-scheduler-arg: bind-address=192.168.4.2
node-taint:
  - node.cilium.io/agent-not-ready:NoExecute
  - node-role.kubernetes.io/control-plane:NoSchedule
server: https://192.168.4.10:6443
tls-san:
  - 192.168.4.10
token: redacted

[dereknola]

The values you are looking for are defined in https://github.com/k3s-io/k3s/blob/master/manifests/coredns.yaml#L125-L130. They are controlled via the manifests for the pods. You will need to override those values with your own using. https://docs.k3s.io/helm#customizing-packaged-components-with-helmchartconfig.

[fmunteanu]

Thank you for the useful information @dereknola. Just some clarification, your Helm example talks about HelmChartConfig kind, but my changed are related to Deployment kind. What would be the proper template format, DeploymentConfig?

Since containers are a list, setting a specific value will delete the other list elements:

apiVersion: helm.cattle.io/v1
kind: DeploymentConfig
metadata:
  name: coredns
  namespace: kube-system
spec:
  template:
    spec:
      containers:
      - name: coredns
        resources:
          limits:
            cpu: null
            memory: 256Mi
          requests:
            cpu: 100m
            memory: 256Mi

What would be the correct template format?

[brandond]

• Ref: https://github.com/k3s-io/k3s/issues/11348#issuecomment-2489253653