[Feature Request] Configure Huge Pages via Custom Node Config

jopalhei commented 1 year ago

Is your feature request related to a problem? Please describe. On our environment we need to make use of Huge Pages, this is because we never want to swap out the Huge Pages, they must be statically allocate. There is nothing other than our application on the worker node that requires Huge Pages, but we must ensure that the Huge Pages are always available to our application.

Currently it is possible to make some Custom Configuration to the AKS nodes following this article : https://learn.microsoft.com/en-us/azure/aks/custom-node-configuration but there is nothing related to Huge Pages, for example we would like to configure the setting vmNrHugepages .

We are currently using a Daemonset to do this, but there is a big problem for us with this approach :

It’s not possible to auto-scale a Nodepool from 0 worker nodes when using DaemonSets to configure Huge Pages. For context, the business case is that we intend to perform vertical auto-scaling of Pods when they require more memory. Essentially using node affinity to move a Pod from one Nodepool to another Nodepool that has a larger memory capacity. In practice, we need to have multiple Nodepools with large amounts of memory readily available to be scaled from 0 to x worker nodes automatically. We cannot have all of our Nodepool set to a min. of 1 node as it would result in wasted resources.
In our experience, it takes significantly longer to prepare and get a new worker node into a ready state, because it has to be configured by the DaemonSet first.

Describe the solution you'd like We would like to have this setting available to be configured as a Custom Node Setting : https://learn.microsoft.com/en-us/azure/aks/custom-node-configuration

Describe alternatives you've considered We are currently using a deamonset to achieve this but it's far from ideal for our case scenario.