F5Networks / k8s-bigip-ctlr

Repository for F5 Container Ingress Services for Kubernetes & OpenShift.
Apache License 2.0
351 stars 193 forks source link

pool members are not updated correctly when nodeportlocal is used #3466

Open ChrisL16 opened 1 week ago

ChrisL16 commented 1 week ago

Setup Details

CIS Version : 2.17.0
Build: f5networks/k8s-bigip-ctlr:latest
BIGIP Version: Big IP 17.1.1.3
AS3 Version: 3.50.1
Agent Mode: AS3
Orchestration: Tanzu
Orchestration Version: v1.27.11+vmware.1-fips.1
Pool Mode: nodeportlocal
Additional Setup details: Tanzu / Antrea CNI

Description

We are using CIS with CRDs in Tanzu with nodeportlocal mode. Everything is working fine until we change the number of replicas in a deployment. When we scale up, a pool member is missing; when we scale down, pool members that no longer exist remain. In the logs you can see that all pods are added to or deleted from the CIS cache. Only in the AS3 declaration are pool members missing. Even if we do a roll-up of an app, it happens that the pool members are not updated correctly. When we restart CIS, the configuration is delivered correctly.

Steps To Reproduce

1) configure a deployment with a virtual server crd to publish an app 2) check pool members 3) scale up the deployment replicas 4) check pool members again 5) scale down the deployment replicas 6) check pool members again 7) deploy a new version of the app 8) check pool members

Expected Result

All existing pods are available as pool members.

Actual Result

Pool members are missing or there are some that no longer work.

Diagnostic Information

Args:
      --credentials-directory
      /tmp/creds
      --bigip-partition=lab02-f5ns
      --bigip-url=10.10.87.36
      --custom-resource-mode=true
      --insecure=true
      --log-as3-response=true
      --log-level=AS3DEBUG
      --orchestration-cni=antrea
      --pool-member-type=nodeportlocal
      --share-nodes=true

CIS Logs:
2024/06/25 08:07:18 [DEBUG] Adding Pod 'default/f5-demo-549b555cf9-fz2xk' in CIS cache
2024/06/25 08:07:18 [DEBUG] Adding Pod 'default/f5-demo-549b555cf9-cgztt' in CIS cache
2024/06/25 08:07:18 [DEBUG] Adding Pod 'default/f5-demo-549b555cf9-zwmv6' in CIS cache
2024/06/25 08:07:18 [DEBUG] Adding Pod 'default/f5-demo-549b555cf9-hcrfr' in CIS cache
024/06/25 08:08:16 [DEBUG] Adding Pod 'default/f5-demo-549b555cf9-fz2xk' in CIS cache
2024/06/25 08:08:16 [DEBUG] Deleting Pod 'default/f5-demo-549b555cf9-fz2xk' from CIS cache as it's not referenced by monitored resources
2024/06/25 08:08:17 [DEBUG] Adding Pod 'default/f5-demo-549b555cf9-hcrfr' in CIS cache
2024/06/25 08:08:17 [DEBUG] Deleting Pod 'default/f5-demo-549b555cf9-hcrfr' from CIS cache as it's not referenced by monitored resources
trinaths commented 1 week ago

@ChrisL16 Please share Resource YAML manifest, CIS config to automation_toolchain_pm@f5.com

ChrisL16 commented 6 days ago

@trinaths I have sent the requested infos.