myakscluster

Here are blog articles related to this repository:

To properly setup and secure your AKS cluster, there is a couple of features and components to enable in order to properly setup your Security Posture, here is the list:

Basic - without cost:

[ ] Managed Identity #62
[X] Azure Lock #21
[X] kured #13
[X] Disable K8S Dashboard #24
[X] Network Policy with Calico #9
[ ] AAD #10
[ ] Azure Policy for AKS #11
[X] Azure KeyVault for Azure pipelines #3

Advanced - with cost:

[ ] Uptime SLA #66
[X] Private AKS cluster + Private ACR + Jumpbox with Azure Private Endpoint #57
- Alternatively: Use Authorized IP addresses with AKS and Azure Service Endpoint with ACR.
[ ] Limit Egress Traffic with Azure Firewall #16

Complementary Azure services to leverage:

Azure Advisor for AKS
Azure Security Center for ACR
Azure Security Center for AKS

Architecture diagram

Provisioning

Prerequisites

Install Azure CLI
Install Terraform
- Optional if you would like to leverage Terraform instead of Azure CLI.
Install Azure ConnectedK8s and KubernetesConfiguration CLI extensions
- Optional if you would like to use Azure Arc enabled Kubernetes.

Configuration pre-provisioning

az login
#az account list -o table
#az account set -s <subscriptionId>

export LOCATION='canadacentral'
export NODES_COUNT=3
export NODE_SIZE='Standard_DS2_v2'
export ZONES=false

./run-pre-checks.sh

randomSuffix=$(shuf -i 1000-9999 -n 1)
export AKS=FIXME$randomSuffix
export K8S_VERSION=$(az aks get-versions \
  -l $LOCATION \
  --query "orchestrators[?isPreview==null].orchestratorVersion | [-1]" \
  -o tsv)

Provisioning Option 1: Azure CLI

cd cli
./create-aks-cluster.sh

FYI, current issues/workarounds with Azure CLI:

az aks create|nodepool add --labels - labels ignored if prefix contains '.'
- Need to manually apply the kubernetes.azure.com/mode=user k8s label on each user node.

Provisioning Option 2: Terraform

cd tf
sudo terraform init
terraform plan \
  -var aks_name=$AKS \
  -var k8s_version=$K8S_VERSION \
  -var location=$LOCATION \
  -var aks_node_count=$NODES_COUNT \
  -var aks_node_size=$NODE_SIZE
terraform apply \
  -auto-approve \
  -var aks_name=$AKS \
  -var k8s_version=$K8S_VERSION \
  -var location=$LOCATION \
  -var aks_node_count=$NODES_COUNT \
  -var aks_node_size=$NODE_SIZE

Configuration post-provisioning

You need to connect to the Jumpbox VM via the Bastion host and run the commands below:

az login \
  --service-principal \
  -u FIXME \
  -p FIXME \
  --tenant-id FIXME
az aks get-credentials \
  -g $RG \
  -n $RG
./configure-aks-cluster.sh

Pricing estimation

AKS
- $0.10 per cluster per hour.
Virtual Machines related to AKS worker nodes
- Estimation: For 2 Node pools X 3 Standard_DS2_v2 Ubuntu worker nodes, ~ $xx USD/month
- To decrease the cost: you may want to leverage Azure Reserved VMs Instances to reduce the cost of your AKS worker nodes
Managed Disks (related to the AKS's Nodes OS Disk)
- Estimation: ~ $xx USD/month
Log Analytics Workspace (related to the Azure Monitor for Containers add-on)
- Estimation: For xx GB/month, ~ $xx USD/month
- To decrease the cost: you may want to leverage logs filtering, retention by data type or New capacity-based pricing option.
Standard Load Balancer
- Estimation: TODO
- To decrease the cost: you may want to leverage Basic Load Balancer instead which is free but you won't be able to use Availability Zones nor VNET Peering features
Azure Firewall
- Estimation: TODO
Public IP
- Related to the SLB + FW
- Estimation: TODO
Bandwidth (related to Availability Zones)
- Estimation: TODO
- Note: you may want to leverage PodAffinity to avoid cross-zones communications with Pods too chatty
Private Endpoint
- Estimation: For 2 Private Endpoints (AKS + ACR)
Bastion - TODO

Other considerations:

Regulatory compliance
Security hardening in AKS virtual machine hosts
Regularly update to the latest version of Kubernetes
Audit logging in AKS
Azure AD PIM
Azure DDOS Protection
Azure Front Door - TODO

mathieu-benoit / myakscluster

readme