'osr_openshift_ingress_canary_canary'; operand 'http-uri' with condition 'equals' requires at least 1 value

mikeoleary commented 1 month ago

Setup Details

CIS Version : 2.18
Build: f5networks/k8s-bigip-ctlr:latest
AS3 Version: 3.53
Orchestration: openshift 4.16

Description

The default Route object called canary in the namespace openshift_ingress_canary does not have a spec.path configured.

This error is thrown by CIS when CIS is deployed into Openshift watching all clusters and managing routes.

Freshly deployed cluster. Nothing else deployed but CIS.

Additional Routes created by customer are not processed and exposed via BIG-IP, because CIS is receiving a 422 error.

2024/10/17 15:31:19 [DEBUG] Admitted Route -  svc1-route-passthrough
2024/10/17 15:31:22 [DEBUG] [AS3] Posting AS3 Declaration
2024/10/17 15:31:22 [DEBUG] [AS3] posting request to [https://10.88.8.204/mgmt/shared/appsvcs/declare/](https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2F10.88.8.204%2Fmgmt%2Fshared%2Fappsvcs%2Fdeclare%2F&data=05%7C02%7CMI.OLEARY%40f5.com%7Cb9b54817b09e48d5ba7808dceec24370%7Cdd3dfd2f6a3b40d19be0bf8327d81c50%7C0%7C0%7C638647765340759956%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=AhNnZkwFqycI%2BvVEZ9CRsYXGA0VWLyV4%2FU%2FwtoESCaE%3D&reserved=0)
2024/10/17 15:31:29 [ERROR] [AS3] Big-IP Responded with error code: 422
2024/10/17 15:31:29 [DEBUG] [AS3] Raw response from Big-IP: map[code:422 declaration:map[class:ADC controls:map[archiveTimestamp:2024-10-17T15:31:26.233Z class:Controls userAgent:CIS/v2.18.0 K8S/v1.29.7+4510e9c] id:urn:uuid:85626792-9ee7-46bb-8fc8-4ba708cfdc1d label:CIS Declaration remark:Auto-generated by CIS schemaVersion:3.52.0 updateMode:selective] results:[map[code:422 declarationId:urn:uuid:85626792-9ee7-46bb-8fc8-4ba708cfdc1d host:localhost message:declaration failed response:01071709:3: Policy '/openshift_partition/Shared/openshift_insecure_routes', rule 'osr_openshift_ingress_canary_canary'; operand 'http-uri' with condition 'equals' requires at least 1 value. runTime:2720 tenant:openshift_partition]]]
2024/10/17 15:31:29 [DEBUG] [AS3] Preparing response message to response handler for arp and fdb config
2024/10/17 15:31:29 [DEBUG] [AS3] Sent response message to response handler for arp and fdb config
2024/10/17 15:31:29 [DEBUG] [AS3] Error handling for event statusUnprocessableEntity
2024/10/17 15:31:29 [DEBUG] [CORE] Updating Route Admit Status

Steps To Reproduce

CIS throwing errors under the following conditions

deployed OCP cluster 4.16
deployed CIS via OperatorHub (CIS v2.18)
CIS is watching all namespaces
manage-routes=true

Expected Result

CIS can be installed and we can start deploying apps.

Actual Result

Workarounds:

1) I have deployed CIS watching select namespace to avoid this error successfully. 2) I have also edited this default Route object and added a spec.Path attribute of "/". CIS will then recover.

Diagnostic Information

<Configuration files, error messages, logs>
Note: Sanitize the data. For example, be mindful of IPs, ports, application names and URLs
Note: The following F5 article outlines the information required when opening an issue.
https://support.f5.com/csp/article/K60974137

Observations (if any)

trinaths commented 1 month ago

Created [CONTCNTR-4945] for internal tracking.

pmahdev commented 4 weeks ago

To troubleshoot this issue we might need more info, Would you like to share us the configuration snippet?

Able to reproduce the issue but need some more context on how are we exposing the canary route object towards the CIS.

mikeoleary commented 4 weeks ago

Hi @pmahdev - I'm glad you can reproduce.

We're not deliberately exposing the canary route. We're just deploying a fresh OCP 4.16 cluster in AWS using the aws installer (UPI method) and then installing CIS with the argument --manage-routes=true.

If we do not limit the namespaces CIS can watch, then this pre-existing Route object is picked up and processed by CIS. Does that make sense?

pmahdev commented 3 weeks ago

After analysis and other scenarios observation this is a configuration issue, where the CIS must be configured with the required route label so that any route resources exposed to the CIS. Need to document this accordingly for ease

Like this, CIS filters route resources by specific labels to process only those that matches.

Additionally we have to support the subdomain for the routes which can be tracked in the backlog [CONTCNTR-4960]

mikeoleary commented 3 weeks ago

@pmahdev thanks for your clarification. So, when installing CIS in OpenShift and when managing Routes, we should use route-label=some_value_here and then label the Routes we want CIS to process. This is great to know.

May I suggest that when documentation is updated, we ALSO update the OpenShift operator so that this config parameter is included by default? I think this would make it much harder for customers to overlook or forget.

F5Networks / k8s-bigip-ctlr