Support customizing tolerations

[andyndang]

/feature

Is your feature request related to a problem? Please describe. We have nodes with taints and the driver is not being scheduled there

Describe the solution you'd like in detail Similiar to the eks-pod-identity-agent configuration that allows taints

[dlakhaws]

Thank you for the feature request. This is something we are tracking, we are planning to publish a public roadmap shortly and will provide updates on the timeline for adding this feature.

[dlakhaws]

Custom tolerations are now supported. This will be a part of the next driver release.

[yitsehong]

Where is the managed addons using value? I am encounter the problem by using taint. Other Managed CSI like EBS or EFS have tolerations for all node

      tolerations:
      - operator: Exists

Would you consider put the tolerations in default value?

[dlakhaws]

Yes, tolerations will be a default value (set to False) in our next release of the driver. We will notify in this thread when that's available for EKS Addon.

[jjkr]

Custom tolerations has been released in v1.4.0. It is currently available via helm install and will be rolling out as an eks addon next week.

[olsib]

This is not working with the EKS Addon. I tried the following:

...
      resolve_conflicts        = "OVERWRITE"
      configuration_values = jsonencode({
        tolerations = [{
          "operator" = "Exists"
        }]
      })
...

│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "xxx"
│   },
│   AddonName: "aws-mountpoint-s3-csi-driver",
│   ClusterName: "cluster-name",
│   Message_: "ConfigurationValue provided in request is not supported: Json schema validation failed with error: [$.tolerations: is not defined in the schema and the schema does not allow additional properties]"
│ }
│ 

I tried v1.4.0-eksbuild.1 & v1.5.1-eksbuild.1

aws eks describe-addon-configuration --addon-name aws-mountpoint-s3-csi-driver --addon-version v1.5.1-eksbuild.1
{
    "addonName": "aws-mountpoint-s3-csi-driver",
    "addonVersion": "v1.5.1-eksbuild.1",
    "configurationSchema": "{\"$schema\":\"https://json-schema.org/draft/2019-09/schema\",\"additionalProperties\":false,\"description\":\"No configurable parameters for Mountpoint for S3 CSI Driver\",\"type\":\"object\"}"
}

aws eks describe-addon-configuration --addon-name aws-mountpoint-s3-csi-driver --addon-version v1.4.0-eksbuild.1
{
    "addonName": "aws-mountpoint-s3-csi-driver",
    "addonVersion": "v1.4.0-eksbuild.1",
    "configurationSchema": "{\"$schema\":\"https://json-schema.org/draft/2019-09/schema\",\"additionalProperties\":false,\"description\":\"No configurable parameters for Mountpoint for S3 CSI Driver\",\"type\":\"object\"}"
}

@jjkr Can you please reopen ?

[barryib]

The daemonset should all taints by default. This is the case for EFS CSI driver, EBS CSI driver and mostly all critical daemonsets. I would suggest to even follow the same implementation than EFS and EBS CSI drivers.

In our case, we use Karpenter by exemple and taint/tolarations to separate workloads. EBS CSI drivers and EFS CSI drivers are working well. But getting toleration issues for S3 CSI driver.

Also related to:

• https://github.com/aws/containers-roadmap/issues/2363
• https://github.com/aws/containers-roadmap/issues/2220