How can we reach the cpx-ingress-controller on 443 ?

[mleneveut]

Hello,

We used the helm chart nsic-v1.43.7 to deploy an netscaler-ingress-controller and netscaler-cpx-with-ingress-controller linked to a Netscaler.

We want to use SSL between the Netscaler and the CPX inside the Kubernetes (security prerequisites from our sec team)

How could we do that ? The cpx pod doesn't seem to listen to 443 at all.

Thanks.

[subashd]

hi @mleneveut , Ports 80 and 443 of CPX service are exposed. While deploying in the Kubernetes, you can expose the service of CPX as type NodePort. The NetScaler Ingress Controller ( NSIC) configuring the NetScaler will automatically updates the IP and Port for CPX ( NodeIP and NodePort of the node where CPX pod is running), if you provided CPX service as backend service in Ingress YAML.

To expose service of CPX as NodePort: https://github.com/netscaler/netscaler-helm-charts/tree/master/netscaler-cpx-with-ingress-controller#netscaler-cpx-servicetype-nodeport

Sample Ingress for configuring NetScaler.

kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name:  netingress
  annotations:
    ingress.citrix.com/frontend-ip: <VIP>
spec:
  ingressClassName: <ingressclass>
  tls:
    - hosts:
        - <host>
       secretName: <TLS Secret >
  rules:
    - host: <host>
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: <service name of CPX> 
                port:
                  number: 443

To better understand the requirement, could you please fill the questionnaire? Also you can reach out us on: netscaler-appmodernization@cloud.com.

[mleneveut]

Thanks for your answer @subashd

I filled the questionnaire.

We would like to call this Ingress on 443 so that the request from Netscaler to CPX is secured :

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    ingress.citrix.com/frontend-ip: 10.x.x.15
    ingress.citrix.com/secure-service-type: ssl
    ingress.citrix.com/secure_backend: "True"
  name: cpx-ingress
  namespace: netscaler
spec:
  ingressClassName: tier-1-vpx
  rules:
  - host: testapp.mycomp.com
    http:
      paths:
      - backend:
          service:
            name: netscaler-cpx-with-ingress-controller-cpx-service
            port:
              number: 443
        path: /
        pathType: Prefix
  tls:
  - secretName: testapp-tls-secret

But the netscaler-cpx-with-ingress-controller-cpx-service then tries to reach the netscaler-cpx-with-ingress-controller pod on port 443 and it doesn't work, as this pod only listens to 80. We have a connection reset when calling https://testapp.mycomp.com

The netscaler-cpx-with-ingress-controller-cpx-service is exposed with NodePort :

spec:
  clusterIP: 10.90.21.186
  clusterIPs:
  - 10.90.21.186
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: http
    nodePort: 30868
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    nodePort: 31963
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: netscaler-cpx-with-ingress-controller
  sessionAffinity: None
  type: NodePort

To have a secured connection between Netscaler and CPX we want to use the 31963 port in Netscaler (the service group member use the nodePort 31963)

If we set the CPX ingress to :

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    ingress.citrix.com/frontend-ip: 10.x.x.15
    ingress.citrix.com/secure-service-type: ssl
    ingress.citrix.com/secure_backend: "True"
  name: cpx-ingress
  namespace: netscaler
spec:
  ingressClassName: tier-1-vpx
  rules:
  - host: testapp.mycomp.com
    http:
      paths:
      - backend:
          service:
            name: netscaler-cpx-with-ingress-controller-cpx-service
            port:
              number: 80
        path: /
        pathType: Prefix
  tls:
  - secretName: testapp-tls-secret

all is green in Netscaler but it uses the nodePort 80:31357/TCP.

What configuration would create all Netscaler resource as SSL and using a ServiceGroup Member using 443:30323/TCP NodePort to enter the cluster ?

[subashd]

hi @mleneveut, Could you please share the log from the NetScaler ingress controller which is configuring the NetScaler? You can mail to subash.dangol@cloud.com

[mleneveut]

Thank you very much @subashd for you help during the call.

It's working now with this configuration :

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: testapp-ingress
  namespace: default
spec:
  ingressClassName: tier-2-to-cpx
  rules:
  - host: testapp.mycomp.com
    http:
      paths:
      - backend:
          service:
            name: testapp-service
            port:
              number: 80
        path: /
        pathType: Prefix
  tls:
  - secretName: testapp-default-tls-secret

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    ingress.citrix.com/frontend-ip: 10.3.8.15
    ingress.citrix.com/secure_backend: "True"
  name: testapp-cpx-ingress
  namespace: netscaler
spec:
  ingressClassName: tier-1-to-vpx
  rules:
  - host: testapp.mycomp.com
    http:
      paths:
      - backend:
          service:
            name: netscaler-cpx-with-ingress-controller-cpx-service
            port:
              number: 443
        path: /
        pathType: Prefix
  tls:
  - secretName: testapp-tls-secret

So the key was to add a TLS secretName to the testapp ingress.

What could have lead us to the wrong setting is, I think, a mistake in the dual-tiers example : https://docs.netscaler.com/en-us/netscaler-k8s-ingress-controller/deploy/two-tier-deployment

I think that 443 could not work as the service is only listening to 80 :

spec:
  ingressClassName: colddrink
  defaultBackend:
    service:
    name: frontend-colddrinks
    port:
        number: 443
  ssl_tcp:
  - secretName: "colddrink-secret"

So it should be number: 80, and we would have understand that the SSL offloading was done at the CPX level when the TLS secretName is set in the app ingress.

[subashd]

@mleneveut Will review and update the document.