search

noi-techpark / infrastructure-v2

Open Data Hub Infrastructure v2 Repository
1 stars 0 forks source link

feat: introduce cluster autoscaler #49

Closed christian-roggia closed 11 months ago

christian-roggia commented 11 months ago

This pull request introduces the AWS Cluster Autoscaler to the Kubernetes cluster. Additionally, it establishes the initial limits and requests for Helm resources. It's important to note that the AWS autoscaler considers during planning requested resources rather than the actual CPU utilization of the node.

Additionally, updates have been made to the Helm deployment documentation to address missing secrets fetching for RabbitMQ and MongoDB. Furthermore, adjustments to the Kubernetes dashboard and metrics server configuration have been implemented. Users can now observe CPU and RAM utilization through the dashboard graphs.

NOTE FOR REVIEWERS

To fully leverage the AWS autoscaler, it's important to set appropriate resource limits and requests for your collectors. This action will trigger the autoscaler mechanism. Specifically, for spreadsheet collectors, I suggest a minimum of 250Mi and 100m requested.

clezag commented 11 months ago

Thanks for the PR!

Regarding the secrets, did you encounter any issues that made you add the explicit references? IIRC, both those secrets for mongo and rabbit are actually created by the default helm chart configurations themselves with random values, so they wouldn't exist the moment you first install the chart. Feel free to correct me on that.

christian-roggia commented 11 months ago

You are right, they do not exist during new installations but they are required for upgrades, I think they will go away once the CI/CD is implemented.