Auto scaling

[nickumia-reisys]

Related to https://github.com/GSA/datagov-deploy/issues/3669

Reference: https://karpenter.sh/v0.8.0/getting-started/getting-started-with-terraform/

New Additions:

• Deploy Karpenter to utilize AWS Autoscaling capabilities
• Add tags to EKS and VPC to allow Karpenter to find relevant resources
• Add IAM Policies for Karpenter to create new EC2 instances
• Add Karpenter provisioner that is used to create the new EC2 instances

Updates:

• Very important: cluster_endpoint_private_access = true is necessary to get Karpenter to watch resources

[nickumia-reisys]

There's a problem with the kubernetes manfest, it keeps giving this error and I can't figure out the cause. The provider version seems compatible (v2.9) and the manifest is able to apply with kubectl so I'm not sure what the issue is.

[nickumia-reisys]

As to last CI run, it seems like this is awfully suspicious.

[mogul]

Ready to remove draft status?

[nickumia-reisys]

@mogul Not particularly, the switch from null_resource.local_exec to kubernetes_manifest didn't work. The kubernetes_manifest is giving dependency issues even though we are depending on cluster-functional and karpenter.

During the CI run, it tried to apply the kubernetes_manifest almost immediately and I'm not sure how to prevent that,

Doing research, others have had to do a two-apply solution which won't work for us, so should we just go back to the local-exec? Or do you see something wrong with my kubernetes_manifest implementation?

[mogul]

I think the answer will ultimately be to put things that create CRDs in a separate module from things that use those CRDs, which will be possible once this CSB change allows us to use directory paths naturally in the brokerpak HCL.

Until then, I think we should try this other workaround which wraps the CRD usage in a generic helm_release. (That way we don't have to revert back to passing values between our modules.)