This project contains Terraform configuration files to provision infrastructure components required to deploy SAS Viya platform products on Microsoft Azure Cloud.
This gives the user the ability to choose the use the local temporary storage available to some nodes for the Operating system, and or the Kubelet.
The benefit of doing this is the local temporary disk is typically a high performance type of storage medium. If the kubelet is set to use this fast storage, then local emptyDir volume can be used for workloads such as SASWORK. Previously many have used alternatives such as hostPath access methods to this storage which is not often permitted by security policies.
OS Disk type can be either Managed or Ephemeral.
Kubelet Disk type can be either OS or Temporary.
The default behaviour for the option, and when not set, is to use Managed and OS for the OS and Kubelet respectively. When set to Temporary and Ephemeral the local temporary storage will be used if available on the node.
Test 1
Original State. The example tfvars file contains an OS disk of 200GB. This means the kubelet will share this managed disk with the OS, placing both the kubernetes emptyDir volumes, and container image storage on the same disk as the OS.
Name OsDiskType KubeletDiskType
--------- ------------ -----------------
system Managed OS
compute Managed OS
stateless Managed OS
cas Managed OS
stateful Managed OS
Test 2
Functionality added but nothing added to tfvars file
Name OsDiskType KubeletDiskType
--------- ------------ -----------------
system Managed OS
stateful Managed OS
stateless Managed OS
compute Managed OS
cas Managed OS
Test 3
Compute node pool changed to VM which has temporary storage, Standard_E16ds_v5, os disk reduced to 80gb, and both OS and Kubelet disk type switched to their non-standard options
Name OsDiskType KubeletDiskType
--------- ------------ -----------------
system Managed OS
stateful Managed OS
stateless Managed OS
cas Managed OS
compute Ephemeral Temporary
Notice the OS has carved off the 80GB from the (as advertised) 600GB disk. And the kubelet has the remaining space.
Compute OS type set to ephemeral, and kubelet set to OS disk
Name OsDiskType KubeletDiskType
--------- ------------ -----------------
system Managed OS
compute Ephemeral OS
stateful Managed OS
stateless Managed OS
cas Managed OS
It looks like both kubelet and the OS use the same OS disk size, squeezing it all into the 80GB which is carved off from the local temporary disk.
root@aks-compute-34891476-vmss000000:/# ls /mnt
DATALOSS_WARNING_README.txt lost+found
Test 5
Set the OS Disk to Managed, and the Kubelet disk to Temporary
Name OsDiskType KubeletDiskType
--------- ------------ -----------------
system Managed OS
stateful Managed OS
stateless Managed OS
cas Managed OS
compute Managed Temporary
We can see that kubelet is on sdb1, and it has almost all of the local temporary disk for use.
Fixes #363
This PR provides functionality to set the OS Disk Type and Kubelet Disk Type options as documented in: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/kubernetes_cluster_node_pool#kubelet_disk_type
This gives the user the ability to choose the use the local temporary storage available to some nodes for the Operating system, and or the Kubelet. The benefit of doing this is the local temporary disk is typically a high performance type of storage medium. If the kubelet is set to use this fast storage, then local emptyDir volume can be used for workloads such as SASWORK. Previously many have used alternatives such as hostPath access methods to this storage which is not often permitted by security policies.
OS Disk type can be either Managed or Ephemeral. Kubelet Disk type can be either OS or Temporary. The default behaviour for the option, and when not set, is to use Managed and OS for the OS and Kubelet respectively. When set to Temporary and Ephemeral the local temporary storage will be used if available on the node.
Test 1
Original State. The example tfvars file contains an OS disk of 200GB. This means the kubelet will share this managed disk with the OS, placing both the kubernetes emptyDir volumes, and container image storage on the same disk as the OS.
Test 2
Functionality added but nothing added to tfvars file
Test 3
Compute node pool changed to VM which has temporary storage,
Standard_E16ds_v5
, os disk reduced to 80gb, and both OS and Kubelet disk type switched to their non-standard optionsNotice the OS has carved off the 80GB from the (as advertised) 600GB disk. And the kubelet has the remaining space.
Test 4
Compute OS type set to ephemeral, and kubelet set to OS disk
It looks like both kubelet and the OS use the same OS disk size, squeezing it all into the 80GB which is carved off from the local temporary disk.
Nothing in /mnt
Test 5
Set the OS Disk to Managed, and the Kubelet disk to Temporary
We can see that kubelet is on sdb1, and it has almost all of the local temporary disk for use.