Use gp3 as default EKS cluster storage class

[zack-is-cool]

Persona

Maintainer

Description

use gp3 as default storage class for the eks cluster

Here's an example of doing it via blueprints (what we are using now) https://github.com/aws-ia/terraform-aws-eks-blueprints/blob/35d131775b0c2f9235fa880bf171e935b8b50ce3/examples/stateful/main.tf#L212-L259

Blueprints has the ability to use the marketplace addon (through TF) or apply it via a helm chart (through TF). We are currently using blueprints and using the marketplace addon variable.

marketplace addons for 1.26 see output:

aws eks describe-addon-versions \
--kubernetes-version 1.26 \
--query 'addons[].{MarketplaceProductUrl: marketplaceInformation.productUrl, Name: addonName, Owner: owner Publisher: publisher, Type: type}' \
--output table

-----------------------------------------------------------------------------------------
|                                 DescribeAddonVersions                                 |
+------------------------+-----------------------+--------+------------+----------------+
|  MarketplaceProductUrl |         Name          | Owner  | Publisher  |     Type       |
+------------------------+-----------------------+--------+------------+----------------+
|  None                  |  kube-proxy           |  aws   |  eks       |  networking    |
|  None                  |  aws-guardduty-agent  |  aws   |  eks       |  security      |
|  None                  |  adot                 |  aws   |  eks       |  observability |
|  None                  |  vpc-cni              |  aws   |  eks       |  networking    |
|  None                  |  coredns              |  aws   |  eks       |  networking    |
|  None                  |  aws-ebs-csi-driver   |  aws   |  eks       |  storage       |
+------------------------+-----------------------+--------+------------+----------------+

Use Case

Saves money. Newer storage class.

Impact

Doesn't save money?

Completion

Have ability to use gp3 storage class by default

Additional Context

more info here: https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/main/docs/amazon-eks-addons.md

Run this for the available config options:

# get the addon version
aws eks describe-addon-versions \
  --kubernetes-version 1.27 | code -

# get the configuration schema based on addon-version and name
aws eks describe-addon-configuration \
  --addon-name aws-ebs-csi-driver \
  --addon-version v1.21.0-eksbuild.1 \
  --query 'configurationSchema' \
  --output text | jq

[ntwkninja]

This PR merged in the base ability to toggle on gp3 as the default storageclass via a single tf var. Additionally, it variabilized the reclaim policy to enable setting retain (default is delete)

Prior to the method seen in the PR, I had successfully used the helm provider / ebs_csi_helm_config to create a gp3 storage class and set it to default; however, I was unable to change the gp2 storage class to be non-default via that method because it wasn't managed-by: Helm

[ntwkninja]

closing as complete per our convo

will open a new issue as needed