feat: Upgrading

[candonov]

What does this PR do?

• upgrading k8s to 1.25
• moved EKS cluster creation to the official tf module
• updated tf providers to use exec plugin
• using SPOT instances and Karpenter for cost efficiency
• added in-cluster Prometheus
• upgraded Crossplane to latest
• upgraded both aws providers to latest
• enabled environment config and functions
• enabled metrics

Motivation

Upgrades and preparation for terraform blueprints v5.

More

• [X] Yes, I have tested the PR using my local account setup (Provide any test evidence report under Additional Notes)

• [ ] Yes, I have added a new example under examples to support my PR

• [ ] Yes, I have updated the docs for this feature

• [ ] Yes, I have linked to an issue or feature request (applicable to PRs that solves a bug or a feature request)

Note:

• Not all the PRs require examples and docs
• We prefer small, well tested pull requests. Please ensure your pull requests are self-contained, and commits are squashed

For Moderators

• [ ] E2E Test successfully complete before merge?

Additional Notes

[csantanapr]

@candonov Getting error when deploying

module.eks_blueprints_kubernetes_addons.module.prometheus[0].module.helm_addon.helm_release.addon[0]: Still creating... [3m0s elapsed]

....
╷
│ Warning: Helm release "prometheus" was created but has a failed status. Use the `helm` command to investigate the error, correct it, then run Terraform again.
│ 
│   with module.eks_blueprints_kubernetes_addons.module.prometheus[0].module.helm_addon.helm_release.addon[0],
│   on .terraform/modules/eks_blueprints_kubernetes_addons/modules/helm-addon/main.tf line 1, in resource "helm_release" "addon":
│    1: resource "helm_release" "addon" {
│ 
╵

╷
│ Error: timed out waiting for the condition
│ 
│   with module.eks_blueprints_kubernetes_addons.module.prometheus[0].module.helm_addon.helm_release.addon[0],
│   on .terraform/modules/eks_blueprints_kubernetes_addons/modules/helm-addon/main.tf line 1, in resource "helm_release" "addon":
│    1: resource "helm_release" "addon" {
│ 

[candonov]

@csantanapr, can you check the status of the Prometheus pods? Why are they not coming up?

[candonov]

@csantanapr I implemented a workaround the terraform helm_release provider. The cluster and add-ons creation is fast now. Please have another look. It is ready to merge if no other issues arise.