search

Azure / aksArc

# Welcome to the Azure Kubernetes Service on Azure Stack HCI repo This is where the AKS-HCI team will track features and issues with AKS-HCI. We will monitor this repo in order to engage with our community and discuss questions, customer scenarios, or feature requests. Checkout our projects tab to see the roadmap for AKS-HCI!
MIT License
109 stars 45 forks source link

Improve the Scale-In functionality of Set-AksHciCluster #115

Open eponerine opened 2 years ago

eponerine commented 2 years ago

Sometimes, you may need to explicitly specify which k8s worker nodes you want to kill while scaling in. Perhaps you have FCM/S2D issues or maybe there's just something wrong with a VM and you don't want to waste time triaging (cattle, not pets).

Currently, Set-AksHciCluster only allows you to specify a count of worker nodes. Presumably there are built-in smarts that pick the best candidate nodes to decommission when you opt to scale-in. It would be awesome to have a new parameter that let you specify the worker nodes to kill (k8s hostname, FCM resource name, whatever). I imagine it would be something like this:

Assuming there are currently 7 Linux Worker Nodes and you want to reduce down to 5

Set-AksHciCluster -linuxNodeCount 5 -linuxNodesToScaleIn @('k8s-cluster-default-linux-nodepool-md-nr9dx-8b48eab5', 'k8s-cluster-default-linux-nodepool-md-kth4x-1db964b8')

The logic during the scale-in would check if NewNodeCount == (OldNodeCount - HostnameCollection.Length) to make sure you aren't specifying too many node names.

Elektronenvolt commented 2 years ago

Hi Ernie

I've checked now available commands at the AksHci module, I would delete damaged nodes with Remove-AksHciNode and expect that new ones get created to get back to the desired cluster state. (I'll try it out on my current installation the next days). Why would you like to scale down in case of damaged nodes?

eponerine commented 2 years ago

That cmdlet is for physical nodes in your Windows Failover Cluster (at least that's how documentation describes it).

I'm referring to k8s worker node VMs.

Specifying the VMs you want to destroy has a bunch of utilities, one example is "the VM is broken due to underlying storage or Hyper-V issue".

Elektronenvolt commented 2 years ago

mmmh.. it's in the AksHci PS module -> I would expect it's about the K8S nodes. Description says Remove a failed node from the Moc stack during a Failure Replacement Unit scenario

I'll try it.. let's see what happens.. 😃

Elektronenvolt commented 2 years ago

Yes, didn't remove any worker - and I found it - the description in the docs is more clear about it: remove-akshcinode
Remove a physical node from your deployment. You can use this command in the case that a physical node fails.

But, since there are nodepools available, does the Set-AksHciCluster -Name <String> -linuxNodeCount <Int32> -windowsNodeCount <Int32> command still work in your setup? I get back: Error: [akshcinodepools.msft.microsoft "clustername-linux" not found].

There is a Remove-AksHciNodePool - may to have it as a parameter on this?