projectcalico / calico

Cloud native networking and network security
https://docs.tigera.io/calico/latest/about/
Apache License 2.0
6.04k stars 1.34k forks source link

Feature request: Support AKS with Calico on Windows nodes #6846

Open olljanat opened 2 years ago

olljanat commented 2 years ago

Current Behavior

Currently it is possible Bring your own CNI to AKS and instruction for that is available in this Calico document. However that same document says that "Windows dataplane is not supported."

Expected Behavior

It would be nice to have Windows also supported or at least discussed in here that what need to happen that it can be supported or explanation why it cannot be.

Possible Solution

I already quicky tested that deploying Calico as HostProcess containers to Windows partly works if these settings:

CNI_BIN_DIR: "c:\\k\\azurecni\\bin"
CNI_CONF_DIR: "c:\\k\\azurecni\\netconf"
CNI_CONF_FILENAME: "10-azure.conflist"

are used in calico-windows.yaml

As partly I mean that calico-node.exe is already running on Windows nodes with that config and pods gets IP addresses from Calico IPAM but Windows nodes are still stuck in NotReady status. However I was not able to figure out that why (at least yet).

In additionally I found that this kind of deployment can be used to get shell access to those Windows nodes:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: debug-windows
  labels:
    app: debug-windows
spec:
  replicas: 1
  selector:
    matchLabels:
      app: debug-windows
  template:
    metadata:
      name: debug-windows
      labels:
        app: debug-windows
    spec:
      hostNetwork: true
      securityContext:
        windowsOptions:
          hostProcess: true
          runAsUserName: "NT AUTHORITY\\Local service"
      hostNetwork: true
      containers:
      - name: debug-windows
        image: mcr.microsoft.com/windows/servercore:ltsc2022
        command:
          - ping
          - -t
          - 127.0.0.1
      nodeSelector:
        "kubernetes.io/os": windows
      tolerations:
      - key: "node.cloudprovider.kubernetes.io/uninitialized"
        operator: "Exists"
      - key: "node.kubernetes.io/not-ready"
        operator: "Exists"
coutinhop commented 8 months ago

@olljanat just a quick update: it seems like there is currently no support for BYO CNI on Windows in AKS (the Azure CNI is installed on Windows nodes regardless of if --network-plugin none is used), se we're blocked on AKS adding support. Once that is done, Calico CNI on Windows nodes should just work in AKS