search

F5Networks / k8s-bigip-ctlr

Repository for F5 Container Ingress Services for Kubernetes & OpenShift.
Apache License 2.0
364 stars 195 forks source link

Calico static routes not updated when nodes added/removed from cluster #3655

Open mikeoleary opened 1 day ago

mikeoleary commented 1 day ago

Setup Details

CIS Version : 2.18.1 Build: f5networks/k8s-bigip-ctlr:latest
Agent Mode: AS3 Orchestration: K8S Pool Mode: Cluster Additional Setup details: Customer has Calico CNI

Description

Submitting on behalf of customer, pls reach out if cust details are required. Customer reports that not all Calico routes show up on BIG-IP when using static routing mode. Customer reports that CIS is not updating routes when Calico issues a new /26 CIDR block to a node.

Customer reports this behavior with CIS 2.18.1

Customer is aware of how to correctly deploy CIS with static-routing-mode and is using --static-routing-mode=true and --orchestration-cni=calico-k8s

Customer reports Calico version as follows:

Here is the Calico Enterprise version info: Cluster Calico Version: v3.26.0 ß Opensource version Cluster Calico Enterprise Version: v3.18.4 ß Calico Enterprise version

Steps To Reproduce

1) Deploy CIS 2.18.1 with a K8s cluster running Calico CNI, and use --static-routing-mode=true and --orchestration-cni=calico-k8s 2) See that multiple static routes are created. In this case, a /26 CIDR block for each node. 3) Add or remove nodes from the cluster. Calico will assign new CIDR blocks for new nodes and remove/reclaim blocks from deleted nodes. 4) CIS does not update the static routes on BIG-IP for added/removed nodes.

Expected Result

CIS should update the static routes on BIG-IP for added/removed nodes.

Actual Result

CIS does not update the static routes as nodes are added or removed.

Diagnostic Information

<Configuration files, error messages, logs>
Note: Sanitize the data. For example, be mindful of IPs, ports, application names and URLs
Note: The following F5 article outlines the information required when opening an issue.
https://support.f5.com/csp/article/K60974137

Observations (if any)

mikeoleary commented 7 hours ago

Update from customer:

It may not be when nodes are added to the cluster, but when Calico assigns an additional /26 CIDR block to an existing node when the pod count requires more IP addresses. Calico will also reclaim that /26 pool once the Pod count goes down and the pool is cleared out.