Controller stops posting changes at runtime when multiple ingress use the same backend

Setup Details

CIS Version : 2.16.1 and 2.17.1
Build: f5networks/k8s-bigip-ctlr:latest
BIGIP Version: Big IP 15.1.8 Build 0.0.7 Final AS3 Version: 3.26.1
Agent Mode: AS3
Orchestration: K8S
Orchestration Version:
Pool Mode: Cluster
Additional Setup details: Kubernetes version: v1.23.8 Calico v3.24.3

Description

When two ingress resources point to the same service, the controller stops posting any further runtime changes to the F5 Big-IP LTM without showing any reason or error. User modifications do generate new posts to de F5 Big-IP LTM in this situation.

However, if you configure the same scenario but with 2 rules within the same ingress resource, the controller works fine and keeps publishing changes at runtime.

Steps To Reproduce

1) Create an ingress with host ingress1.example.com pointing to service1 2) Create another ingress in the same namespace with host ingress2.example.com pointing to the same service1 3) Delete any pod used by a backend on the F5 from the cluster to recreate it and get a new IP.

Expected Result

The controller should post to BIGIP LTM the new configuration with the new IP for the pod.

Actual Result

The controller detects the change but does not generate any post.

Diagnostic Information

f5_controller_issue.tar.gz

I'm providing the next files:

bigip-conf.txt: Configuration of all elements for the bigip controller.
ok-conf.txt: Configuration of all the elements (ns, pods, svc, ...) to get a workload registered on the BigIP LTM with no issues.
ok_2-16.log / ok_2-17.log: Controller logs (version 2.16.1 and 2.17.1 respectively) in debug mode for the above configuration during the restart of the pod upc-bigip-ctlr-default-backend* so you can see the controller detects and posts the change in the endpoint's ip address.
ok_2-16_pod-changes.log / ok_2-17_pod-changes.log: watch log (version 2.16.1 and 2.17.1 respectively) during the restart of the pod upc-bigip-ctlr-default-backend* so you can see the old and the new ip address of the pods.
error-conf.txt: Configuration of all the elements (ns, pods, svc, ...) to reproduce the error.
error_2-16.log / error_2-17.log: Controller logs (version 2.16.1 and 2.17.1respectively) in debug mode for the above configuration during the restart of the pod upc-bigip-ctlr-default-backend* so you can see the controller detects the change in the endpoint's ip address but it does not post anything.
error_2-16_pod-changes.log / error_2-17_pod-changes.log: watch log (version 2.16.1 and 2.17.1 respectively) during the restart of the pod upc-bigip-ctlr-default-backend* so you can see the old and the new ip address of the pods.
ok-2rules-conf.txt: Configuration of all the elements (ns, pods, svc, ...) where instead of using 2 different ingress we configured only 1 ingress with two rules and the contrller works fine.
ok-2rules_2-16.log / ok-2rules_2-17.log: Controller logs (version 2.16.1 and 2.17.1 respectively) in debug mode for the above configuration during the restart of the pod upc-bigip-ctlr-default-backend* so you can see the controller detects and posts the change in the endpoin's ip address.
ok-2rules_2-16_pod-changes.log / ok-2rules_2-17_pod-changes.log: watch log (version 2.16.1 and 2.17.1 respectively) during the restart of the pod upc-bigip-ctlr-default-backend* so you can see the old and the new ip address of the pods.

Observations (if any)

This issue began in CIS version 2.16.1. We suspect that as a result of the next issue resolution https://github.com/F5Networks/k8s-bigip-ctlr/issues/3322

F5Networks / k8s-bigip-ctlr