Failed to create canary patset, Canary deployment will result in patset error when containing "-" symbol the namespace

[progS1m]

Describe the bug Using canary annotations not working when the namespace name is containing a "-" symbol.

Annotations: ingress.citrix.com/canary: true ingress.citrix.com/canary-by-header: X-Canary ingress.citrix.com/canary-by-header-value: preview ingress.citrix.com/canary-weight: 0

Namspace: namespace: my-namespace

Unfortunately we cannot easily change all our namespaces name. Since we have to many of them! And most of them containing "-" symbols!

To Reproduce Deploy the sample like described here, but in a namespace containing a "-" symbol in its name. https://docs.netscaler.com/en-us/netscaler-k8s-ingress-controller/canary/canary.html#simplified-canary-deployment-using-ingress-annotations

Version of the NetScaler Ingress Controller 1.39.6 Version of MPX/VPX/CPX NS 13.1 51.15

Expected behavior

Either the Netscaler should accept "-" symbols in patset or the CIC should convert dashes to underlines. The second option would be probably the faster workaround!

Logs

kubectl logs

2024-03-26 08:37:15,674 - ERROR - [nitrointerface.py:create_canary_patset:7887] (MainThread) Failed to create canary patset k710p_my-namespace_myingressname_Canary_Header_Values_0. Reason (3856, 'Invalid policy entity name: names must begin with an ASCII alphabetic character or underscore and must contain only ASCII alphanumerics or underscores; words reserved for policy use may not be used', 'ERROR')

Probably relates to: https://github.com/netscaler/netscaler-k8s-ingress-controller/issues/547

[ankits123]

@progS1m since '-' is used as a separator in the controller naming convention it is giving above mentioned error. We will discuss your requirement and get back with a update.

[progS1m]

@ankits123 noticed the canary is not always creating a patset. In which constellation are they being crated and in which not?

Using multiple values like described here is not working, respectively doesn't result into a patset.

ingress.citrix.com/canary-by-header-value: '["value1","value2","value3","value4"]'

results in:

2024-03-27 17:19:10,370  - DEBUG - [nitrointerface.py:is_alternate_backend_default_policy:2106] (MainThread) Checking for default cspolicy for alternate backend : policyname:k915p-showhttpvars-r744_80_csp_l6456jwuknnqbiddtdqxcav46sd74o7h and rule: (((HTTP.REQ.HOSTNAME.SERVER.EQ("showhttpvars-web-k915.domain.net") && HTTP.REQ.URL.PATH.SET_TEXT_MODE(IGNORECASE).STARTSWITH("/")) || (HTTP.REQ.HOSTNAME.SERVER.EQ("showhttpvars-web-k915.domain.net") && HTTP.REQ.URL.PATH.SET_TEXT_MODE(IGNORECASE).STARTSWITH("/test"))) || (HTTP.REQ.HOSTNAME.SERVER.EQ("showhttpvars-web-k915.domain.net") && HTTP.REQ.URL.PATH.SET_TEXT_MODE(IGNORECASE).STARTSWITH("/foo"))) || (HTTP.REQ.HOSTNAME.SERVER.EQ("showhttpvars-web-k915.domain.net") && HTTP.REQ.URL.PATH.SET_TEXT_MODE(IGNORECASE).STARTSWITH("/bar")) && HTTP.REQ.HEADER("X-Canary").EQ("["value1","value2","value3","value4"]")

https://docs.netscaler.com/en-us/netscaler-k8s-ingress-controller/canary/canary.html#canary-deployment-based-on-the-http-request-header-value

[progS1m]

@ankits123 noticed the canary is not always creating a patset. In which constellation are they being crated and in which not?

Using multiple values like described here is not working, respectively doesn't result into a patset.

ingress.citrix.com/canary-by-header-value: '["value1","value2","value3","value4"]'

results in:

2024-03-27 17:19:10,370  - DEBUG - [nitrointerface.py:is_alternate_backend_default_policy:2106] (MainThread) Checking for default cspolicy for alternate backend : policyname:k915p-showhttpvars-r744_80_csp_l6456jwuknnqbiddtdqxcav46sd74o7h and rule: (((HTTP.REQ.HOSTNAME.SERVER.EQ("showhttpvars-web-k915.domain.net") && HTTP.REQ.URL.PATH.SET_TEXT_MODE(IGNORECASE).STARTSWITH("/")) || (HTTP.REQ.HOSTNAME.SERVER.EQ("showhttpvars-web-k915.domain.net") && HTTP.REQ.URL.PATH.SET_TEXT_MODE(IGNORECASE).STARTSWITH("/test"))) || (HTTP.REQ.HOSTNAME.SERVER.EQ("showhttpvars-web-k915.domain.net") && HTTP.REQ.URL.PATH.SET_TEXT_MODE(IGNORECASE).STARTSWITH("/foo"))) || (HTTP.REQ.HOSTNAME.SERVER.EQ("showhttpvars-web-k915.domain.net") && HTTP.REQ.URL.PATH.SET_TEXT_MODE(IGNORECASE).STARTSWITH("/bar")) && HTTP.REQ.HEADER("X-Canary").EQ("["value1","value2","value3","value4"]")

https://docs.netscaler.com/en-us/netscaler-k8s-ingress-controller/canary/canary.html#canary-deployment-based-on-the-http-request-header-value

Figured out, the patset was introducing in newer CIC builds, where in older builds no patset has been created.

[progS1m]

@ankits123 /usr/src/triton/kubernetes/canary/ingress_canary.py is the file that has the buggy function get_canary_patset_name

Please fix it by adding an unsupported character substitution something like this: namespace = re.sub('[^A-Za-z0-9]+', '',namespace)

I think it's an easy fix. When can we expect a fix? Unfortunately It is important for us to get this addressed very soon.

[pfyod]

@ankits123: could you please share timeline for this fix? We are unable to use canary features because of this bug - blocked for two months already!

[ankits123]

hi @pfyod we would like to discussion some points over mail . Can you please drop us a email at < netscaler-appmodernization@cloud.com > for the same .