Add-on configuration of aws-efs-csi-driver such as topologySpreadConstraints, replicaCount and resources.

[kimsehyoung]

Situation

I've been installing Add-ons using terraform. And I face the problem on configuration of 'aws-efs-csi-driver' addon. I'm trying to configure 'efs-csi-controller' with topologySpreadConstraints, replica..., But it can't be.

Problem

coredns can be configured with replicaCount, resources, topologySpreadConstraints... But, I can set only 'tolerations' , other things can not be for 'efs-csi-controller' of addon.

It can be checked here with CLI. aws eks describe-addon-configuration --addon-name aws-efs-csi-driver --addon-version v1.7.0-eksbuild.1 --query 'configurationSchema' --output text | jq . 'controller' block

'replicaCount' is out of the block

Solution

Move 'replicaCount' into 'controller' block. And add 'resources' and 'topologySpreadConstraints' into 'controller' block.

[jamesorlakin]

+1 for the resources. The default configuration using stunnel to encrypt NFS traffic in the pod combined with the EFS driver container having a CPU request of 0 can result in volumes becoming 'stuck' for IO if the node is suffering CPU starvation. Other pods with non-zero requests get priority. It would be ideal if the configuration provided the ability to set this, as well as possibly having a non-zero request by default regardless.

It so happens the EBS CSI driver sets a request of 30m so I did similar via a patch, but it sadly makes orchestration a challenge via GitOps or IaC:

kubectl patch ds efs-csi-node -n kube-system -p '{"spec": {"template": { "spec": {"containers": [{"name": "efs-plugin", "resources": {"requests": {"cpu": "30m"}}}]}}}}'

[rmul]

+1 for the replicaCount. We have the same issue when trying to set the replicacount.

[cihatgenc]

+1 Same here, replicaCount not working

[JonMerlevede]

+1 on the replica count. I was really surprised to find it defined outside of the properties block.

Being able to put annotations on the service account would also be nice!

[marco-giovannini-73team]

+1 on adding topologySpreadConstraints

[der-eismann]

I'm missing the affinity in the configuration. Why is it so hard to add this when the EBS driver is having all these options already? :confused:

[thpham]

the schema changed again for v2.0.5-eksbuild.1

and replicaCount is still at weird place ! I also suspect having backend validation error with other fields...

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "additionalProperties": false,
  "description": "Configurable parameters for the aws-efs-csi-driver",
  "properties": {
    "controller": {
      "additionalProperties": false,
      "properties": {
        "deleteAccessPointRootDir": {
          "description": "For dynamic provisioning, control whether the access point directory is deleted on unmount",
          "type": "boolean"
        },
        "logLevel": {
          "description": "The log level of the driver's controller service",
          "maximum": 6,
          "minimum": 0,
          "type": "integer"
        },
        "nodeSelector": {
          "default": null,
          "description": "The nodeSelector Schema for the CSI controller service Deployment",
          "properties": {},
          "type": ["object", "null"]
        },
        "podAnnotations": {
          "default": {},
          "description": "The podAnnotations Schema for the CSI controller service Deployment",
          "properties": {},
          "type": "object"
        },
        "tolerations": {
          "default": [{ "key": "", "operator": "Exists" }],
          "items": {},
          "title": "The tolerations Schema for the CSI controller service Deployment",
          "type": "array"
        }
      },
      "type": "object"
    },
    "node": {
      "additionalProperties": false,
      "properties": {
        "logLevel": {
          "description": "The log level of the driver's node service",
          "maximum": 6,
          "minimum": 0,
          "type": "integer"
        },
        "nodeSelector": {
          "default": null,
          "description": "The nodeSelector Schema for the CSI node service Dameonset",
          "properties": {},
          "type": ["object", "null"]
        },
        "podAnnotations": {
          "default": { "example_node_annotation": "annotation" },
          "description": "The podAnnotations Schema for the CSI node service Dameonset",
          "properties": {},
          "type": "object"
        },
        "tolerations": {
          "default": [{ "key": "", "operator": "Exists" }],
          "items": {},
          "title": "The tolerations Schema for the CSI controller service Deployment",
          "type": "array"
        }
      },
      "replicaCount": {
        "description": "The number of replicas for the CSI controller service Deployment",
        "type": "integer"
      },
      "type": "object"
    },
    "useFIPS": {
      "description": "Assert whether or not the driver uses FIPS enabled endpoints",
      "type": "boolean"
    }
  },
  "type": "object"
}

[martin-liu]

@mskanth972 Do you have any insights for a potential fix of this issue?