NSIC deletes other cluster's CS Vservers when NS_NSIPs used

[arasyor]

We installed Netscaler Ingress Controller on two OpenShift clusters with NS_NSIPS enabled because of pod network overlaps. NSIC works fine on Cluster 1 but when we installed controller on Cluster 2 it deletes other cluster's definitions on Netscaler. You can find deletion logs below on Cluster 2.

2024-03-13 11:51:16,045  - INFO - [NSPBRConfig.py:clean_up_all_static_routes:172] (MainThread) StaticRoute: Deleting all static routes added by CIC
2024-03-13 11:51:21,011  - INFO - [nitrointerface.py:_cleanup_ns_cs_apps:1517] (MainThread) ADC-SYNC: Stale CS Vservers to be deleted: {'openshift-10.79.94.55_443_ssl', 'openshift-10.79.94.55_80_http'}
2024-03-13 11:51:21,293  - INFO - [builderpatternCSVSCleaner.py:clean:139] (MainThread) Procesing cs policies cleanup for cs vs openshift-10.79.94.55_443_ssl
2024-03-13 11:51:21,505  - INFO - [builderpatternCSVSCleaner.py:clean:152] (MainThread) CS policy openshift-httpd_8080-tcp_csp_zfzabgfv42ualpzozcueob67d2werc7s unbind with cs vs openshift-10.79.94.55_443_ssl is successful
2024-03-13 11:51:21,561  - INFO - [builderpatternCSVSCleaner.py:clean:154] (MainThread) CS policy openshift-httpd_8080-tcp_csp_zfzabgfv42ualpzozcueob67d2werc7s delete successful
2024-03-13 11:51:21,784  - INFO - [builderpatternCSVSCleaner.py:clean:160] (MainThread) CS action openshift-httpd_8080-tcp_csp_zfzabgfv42ualpzozcueob67d2werc7s delete successful
2024-03-13 11:51:21,784  - INFO - [builderpatternCSVSCleaner.py:clean:167] (MainThread) Finished procesing cs policies cleanup for cs vs openshift-10.79.94.55_443_ssl
2024-03-13 11:51:21,785  - INFO - [builderpatternCSVSCleaner.py:clean:247] (MainThread) Finished procesing responder policies cleanup for cs vs openshift-10.79.94.55_443_ssl
2024-03-13 11:51:21,785  - INFO - [builderpatternCSVSCleaner.py:clean:298] (MainThread) Procesing default LB VS cleanup for cs vs openshift-10.79.94.55_443_ssl
2024-03-13 11:51:21,785  - INFO - [builderpatternCSVSCleaner.py:clean:312] (MainThread) Finished procesing cs policies cleanup for cs vs openshift-10.79.94.55_443_ssl
2024-03-13 11:51:22,117  - INFO - [nitrointerface.py:_handle_certkey_cleanup:6603] (MainThread) ssl cert keys cleanup for openshift-10.79.94.55_443_ssl csvserver completed
2024-03-13 11:51:22,117  - INFO - [builderpatternCSVSCleaner.py:clean:118] (MainThread) Finished processing SSL policies bindings clean request  for cs vs openshift-10.79.94.55_443_ssl
2024-03-13 11:51:22,563  - INFO - [builderpatternLBVSCleaner.py:clean:116] (MainThread) Service group openshift-httpd_8080-tcp_sgp_zfzabgfv42ualpzozcueob67d2werc7s delete successful from LB VS openshift-httpd_8080-tcp_lbv_zfzabgfv42ualpzozcueob67d2werc7s
2024-03-13 11:51:22,727  - INFO - [nitrointerface.py:_handle_lb_cleanup:6673] (MainThread) lbvserver  openshift-httpd_8080-tcp_lbv_zfzabgfv42ualpzozcueob67d2werc7s delete successful for csvserver openshift-10.79.94.55_443_ssl
2024-03-13 11:51:22,890  - INFO - [nitrointerface.py:_handle_cs_cleanup:6653] (MainThread)  csvserver openshift-10.79.94.55_443_ssl delete successful
2024-03-13 11:51:23,085  - INFO - [builderpatternCSVSCleaner.py:clean:139] (MainThread) Procesing cs policies cleanup for cs vs openshift-10.79.94.55_80_http
2024-03-13 11:51:23,085  - INFO - [builderpatternCSVSCleaner.py:clean:167] (MainThread) Finished procesing cs policies cleanup for cs vs openshift-10.79.94.55_80_http
2024-03-13 11:51:23,301  - INFO - [builderpatternCSVSCleaner.py:clean:237] (MainThread) Responder policy openshift-httpd_8080-tcp_csp_4lmqajj3pfwqrq72mim3rdjqrabc4xm6 unbind with cs vs openshift-10.79.94.55_80_http is successful
2024-03-13 11:51:23,358  - INFO - [builderpatternCSVSCleaner.py:clean:239] (MainThread) Responder policy  openshift-httpd_8080-tcp_csp_4lmqajj3pfwqrq72mim3rdjqrabc4xm6 delete successful
2024-03-13 11:51:23,538  - INFO - [builderpatternCSVSCleaner.py:clean:243] (MainThread) Responder action openshift-httpd_8080-tcp_csp_4lmqajj3pfwqrq72mim3rdjqrabc4xm6 delete successful
2024-03-13 11:51:23,539  - INFO - [builderpatternCSVSCleaner.py:clean:247] (MainThread) Finished procesing responder policies cleanup for cs vs openshift-10.79.94.55_80_http
2024-03-13 11:51:23,539  - INFO - [builderpatternCSVSCleaner.py:clean:298] (MainThread) Procesing default LB VS cleanup for cs vs openshift-10.79.94.55_80_http
2024-03-13 11:51:23,539  - INFO - [builderpatternCSVSCleaner.py:clean:312] (MainThread) Finished procesing cs policies cleanup for cs vs openshift-10.79.94.55_80_http
2024-03-13 11:51:23,723  - INFO - [nitrointerface.py:_handle_cs_cleanup:6653] (MainThread)  csvserver openshift-10.79.94.55_80_http delete successful

Helm installation commands on Cluster 1:

helm upgrade --install nsic netscaler/netscaler-ingress-controller \
  --namespace netscaler-ingress-controller \
  --create-namespace \
  --set adcCredentialSecret=nslogin-local \
  --set clusterName=ocptstinf01 \
  --set crds.install=true \
  --set crds.retainOnDelete=false \
  --set defaultSSLCertSecret=nsic-tst-cert \
  --set entityPrefix=openshift \
  --set ingressClass[0]=netscaler \
  --set serviceClass[0]=netscaler \
  --set license.accept=yes \
  --set nodeSelector.key=node-role.kubernetes.io/infra \
  --set nodeSelector.value="" \
  --set nodeWatch=true \
  --set nsIP=10.81.22.10 \
  --set nsSNIPS='["10.79.94.56"]' \
  --set nsVIP=10.79.94.55 \
  --set openshift=true \
  --set optimizeEndpointBinding=true \
  --set routeLabels="netscaler-ingress-controller=true" \
  --set tolerations[0].effect=NoSchedule \
  --set tolerations[0].key=node-role.kubernetes.io/infra \
  --set tolerations[0].operator=Exists \
  --set tolerations[0].value=""

Helm installation commands on Cluster 2

helm upgrade --install nsic netscaler/netscaler-ingress-controller \
  --namespace netscaler-ingress-controller \
  --create-namespace \
  --set adcCredentialSecret=nslogin-local \
  --set clusterName=ocptstgen01 \
  --set crds.install=true \
  --set crds.retainOnDelete=false \
  --set defaultSSLCertSecret=nsic-tst-cert \
  --set entityPrefix=openshift \
  --set ingressClass[0]=netscaler \
  --set serviceClass[0]=netscaler \
  --set license.accept=yes \
  --set nodeSelector.key=node-role.kubernetes.io/infra \
  --set nodeSelector.value="" \
  --set nodeWatch=true \
  --set nsIP=10.81.22.10 \
  --set nsSNIPS='["10.79.92.56"]' \
  --set nsVIP=10.79.92.55 \
  --set openshift=true \
  --set optimizeEndpointBinding=true \
  --set routeLabels="netscaler-ingress-controller=true" \
  --set tolerations[0].effect=NoSchedule \
  --set tolerations[0].key=node-role.kubernetes.io/infra \
  --set tolerations[0].operator=Exists \
  --set tolerations[0].value=""

[subashd]

hi @arasyor , Could you please set unique entityPrefix in both the helm commands (--set entityPrefix="cluster1")?

To enhance our understanding of your Kubernetes/microservices application deployment architecture and provide you with the latest NetScaler capabilities, we kindly request you to fill out Requirement Gathering Questionnaire. Your valuable insights will enable us to better serve your specific needs.

[arasyor]

hi @subashd ,

Problem is resolved when we set unique entityPrefix in helm installations.

Thanks.

[subashd]

Thank you for confirmation @arasyor. I request you to fill out Requirement Gathering Questionnaire. Your valuable insights will enable us to better serve your specific needs.