multicluster CIS with namespace-label is not working correctly

[avinashchundu9]

Setup Details

CIS Version : 2.17.1
Build: f5networks/k8s-bigip-ctlr:latest
BIGIP Version: Big IP v16.1.3.1 AS3 Version: 3.47
Agent Mode: AS3
Orchestration: K8S
Orchestration Version: v1.28.10
Pool Mode: Nodeport

Description

In multicluster CIS deployed in active-active mode with namespace-label defined in values files, we noticed that the Transport server only contains the pool member of the primary cluster. Removing the namespace-label works with out any issue.

Steps To Reproduce

1) Deploy CIS in active-active mode 2) Values file should include a namespace label 3) Create namespace with labels on both clusters 4) Try to create a Transport server and check for pool members.

Expected Result

The pool should have members from both clusters.

Actual Result

Pool members only have members from the primary cluster

values file: log_level: DEBUG namespace_label: "f5cis-enable=true" pool_member_type: auto insecure: true custom-resource-mode: true log-as3-response: true ipam : false multi-cluster-mode: primary extended-spec-configmap: f5-cis/global-spec-config as3-validation: true

Examples using namespace labels will also help the user community.

[avinashchundu9]

Also when namespace label is used primary and secondary CIS entering into split brain and posting declaration independently overriding each other.

[trinaths]

Created [CONTCNTR-4855] for internal tracking.

[alonsocamaro]

@arzzon I have tested https://github.com/F5Networks/k8s-bigip-ctlr/pull/3557 and It didn´t fix it with my configuration. Tested with image quay.io/f5networks/k8s-bigip-ctlr-devel:2.18.1-26-Sept provided by Vivek. Please see next a test with 3 clusters (ocp1, ocp2 and ocp3)

• When using the CIS config cis-config/f5bigipctlr.ocp1.yaml.works (not speciying namespace or namespace-label parameters, the log logs/cis-all-namespaces.log shows no warnings and pool members from 3 clusters are discovered.

• When using the CIS config cis-config/f5bigipctlr.ocp1.yaml.fails (specifying namespace-label), the log logs/cis-namespace-label.log shows the following warnings and the pool members from ocp3 (external) are not discovered:

% egrep -i "(error|warning)" logs/cis-namespace-label.log
2024/09/26 11:08:22 [WARNING] Creating GTM with default bigip credentials as GTM BIGIP Url or GTM BIGIP Username or GTM BIGIP Password is missing on CIS args.
2024/09/26 11:08:26 [WARNING] Ensure Global Extended Configmap is created in CIS monitored namespace
2024/09/26 11:08:26 [WARNING] [MultiCluster] informer not found for namespace: f5bigipctlr while fetching secret for cluster ocp2
2024/09/26 11:08:29 [WARNING] [MultiCluster] informer not found for namespace: f5bigipctlr while fetching secret for cluster ocp3
2024/09/26 11:08:37 [ERROR] Pool Members could not be fetched for service {cigna-route-a ocp3 openshift-ingress} with targetPort 0:443
2024/09/26 11:08:37 [ERROR] Pool Members could not be fetched for service {cigna-route-b ocp3 openshift-ingress} with targetPort 0:443
2024/09/26 11:08:37 [ERROR] Pool Members could not be fetched for service {cigna-route-a ocp3 openshift-ingress} with targetPort 0:443
2024/09/26 11:08:37 [ERROR] Pool Members could not be fetched for service {cigna-route-b ocp3 openshift-ingress} with targetPort 0:443
2024/09/26 11:08:40 [ERROR] Pool Members could not be fetched for service {cigna-route-a ocp3 openshift-ingress} with targetPort 0:443
2024/09/26 11:08:40 [ERROR] Pool Members could not be fetched for service {cigna-route-b ocp3 openshift-ingress} with targetPort 0:443
2024/09/26 11:08:40 [ERROR] Pool Members could not be fetched for service {cigna-route-a ocp3 openshift-ingress} with targetPort 0:443
2024/09/26 11:08:40 [ERROR] Pool Members could not be fetched for service {cigna-route-b ocp3 openshift-ingress} with targetPort 0:443

Note that the pool members for ocp2 are populated yet the following are shown:

2024/09/26 11:08:26 [WARNING] [MultiCluster] informer not found for namespace: f5bigipctlr while fetching secret for cluster ocp2
2024/09/26 11:08:29 [WARNING] [MultiCluster] informer not found for namespace: f5bigipctlr while fetching secret for cluster ocp3

report-issue3535-pull3557.zip