VirtualServer that references a WAF policy which uses a bundle does not recover if the bundle is missing on startup

[shaun-nx]

Describe the bug When deploying a VirtualServer with a WAF policy which uses the apBundle setting, the VirtualServer will remain go into a Warning state if the expected .tgz bundle file is not present. The VirtualServer will subsequently stay in this Warning state even after the .tgz bundle file is put into the expected directory afterwords.

Example warning message:

Events:
  Type     Reason                     Age                    From                      Message
  ----     ------                     ----                   ----                      -------
  Warning  AddedOrUpdatedWithWarning  4m58s (x3 over 4m58s)  nginx-ingress-controller  Configuration for default/webapp was added or updated ; with warning(s): WAF policy default/waf-policy references an invalid or non-existing App Protect bundle /etc/app_protect/bundles/compiled_policy.tgz

To Reproduce Steps to reproduce the behavior:

1. Deploy NIC v3.5.0 with WAF v4.8.1
2. Deploy a Policy of kind waf, with the apBundle option as shown in the Policy Resource docs
3. Deploy a VirtualServer referencing this poilicy.
4. Ensure the bundle is not present on the file system.,
5. Add the bundle to the expected directory after deployment.

Expected behavior The VirtualServer will go from a Warning state to a Valid state when a waf type policy is Valid

Your environment

• Version of the Ingress Controller - release version or a specific commit: 3.5.0
• Version of Kubernetes: 1.29
• Kubernetes platform (e.g. Mini-kube or GCP): GCP
• Using NGINX or NGINX Plus: NGINX Plus

### Tasks
- [ ] Ensure policy kind becomes invalid when `apBundle` is not foung

[github-actions[bot]]

Hi @shaun-nx thanks for reporting!

Be sure to check out the docs and the Contributing Guidelines while you wait for a human to take a look at this :slightly_smiling_face:

Cheers!

[brianehlert]

This appears to be a workflow or order of operations problem. If a bundle is referenced in a config, it must be present. The question comes if or how the system responds to that.

[brianehlert]

Simply placing the bundle in the correct place, after the fact, does not trigger a reload and thus re-processing of the objects. Some configuration change actions needs to happen to cause the K8s API watcher to trigger and tell NIC to re-process the objects and therefore identify and load the tgz.

NIC should not implement a file watcher. The implemented workflow depends on a YAML resource being modified to trigger the reload. This is all auditable.