Can't see any doco or examples for using existing LB

[ajhindle]

Hi I'd like to use an existing LB that I created previously outside k8s, and tell an ingress to use it i.e. without k8s creating the LB. This OCI Native Ingress Controller doco indicates that I can do it by making an annotation in the ingress.

Do I still need an ingressclass and ingressparameters for this? Please provide an example for my use case.

Thanks

[Inbaraj-S]

Hi @ajhindle , Thank you for reaching out. For your use case, you would need to specify the ingressClassparameter with your existing LB config. sample:

apiVersion: "ingress.oraclecloud.com/v1beta1"
kind: IngressClassParameters
metadata:
  name: ingressparms-cr-test
spec:
  compartmentId: "ocid1.compartment.oc1..aaaaaaaa--<your-compartment-id>"
  subnetId: "ocid1.subnet.oc1.phx.aaaaaaaa--<your-lb-subnet-id>"
  loadBalancerName: "native-ic-lb"
  isPrivate: false
  maxBandwidthMbps: 400
  minBandwidthMbps: 100

Also, you would need an ingressclass with your previously created LB OCID handy:

Sample:

apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: default-ingress-class
  annotations:
    oci-native-ingress.oraclecloud.com/id: "<add-existing-LB-OCID-here>"
    ingressclass.kubernetes.io/is-default-class: "true"

spec:
  controller: oci.oraclecloud.com/native-ingress-controller
  parameters:
    scope: Namespace
    namespace: test
    apiGroup: ingress.oraclecloud.com
    kind: ingressclassparameters
    name: ingressparms-cr-test

[ajhindle]

Thanks @Inbaraj-S I followed your example and was able to make the ingress create a backend to be used by my existing LB. I'm trying to add TLS now as per the doco.
Is it only possible to use end-to-end TLS with this controller? i.e. cannot have listener use 443, terminate TLS, and backend uses port 80. If e2e is the only option, how can I enable my simple demo nginx backend deployment to use the right certificate? Can it be the same certificate as the listener uses?
Usually I use TLS termination, not e2e. Further detail: for TLS config, I am using "Option 2: You obtain certificate from the Certificates service" where the cert is coming from Certificates service in OCI.

My current example ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: native-ic-ingress
  namespace: oci-ing-test
  annotations:
    oci-native-ingress.oraclecloud.com/id: <my lb ocid>
    oci-native-ingress.oraclecloud.com/certificate-ocid: <my cert ocid>
    oci-native-ingress.oraclecloud.com/healthcheck-protocol: "HTTP"
    oci-native-ingress.oraclecloud.com/healthcheck-port: "80"
    oci-native-ingress.oraclecloud.com/healthcheck-path: "/"
    oci-native-ingress.oraclecloud.com/healthcheck-return-code: "200"
    oci-native-ingress.oraclecloud.com/healthcheck-force-plaintext: "true"
spec:
  ingressClassName: native-ic-ingress-class
  rules:
    - host: "my-test.hello.io"
      http:
        paths:
          - pathType: Prefix
            path: /app1
            backend:
              service:
                name: web-service
                port:
                  number: 443

[Inbaraj-S]

@ajhindle TLS termination at LB is added in recent PR, #44 Till the next release comes in, you have to use the same certificate to process the TLS requests from LB on the backend.

[ajhindle]

Thanks @Inbaraj-S Can two (or more) k8s clusters both hosting this ingress controller point at an existing LB, and control their own backends independently?

[Inbaraj-S]

@ajhindle You can do it, but you shouldnt. Each of the ingress controller might undo what the other instance is doing since the ingress resource is cluster specific and other ingress controller might not be aware of these and end up cleaning the LB with unwanted backends/routes/listeners.

[ajhindle]

@Inbaraj-S RE: TLS termination at LB I see that PR #44 is merged into main now. I pulled from main and updated my cluster. I added the new annotation oci-native-ingress.oraclecloud.com/backend-tls-enabled: "false" to my ingress (see below). My current goal is to make it work for an existing LB created separately, and have TLS termination at the LB.

Does the below ingress and service look right? Have I missed something?

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: native-ic-ingress
  namespace: oci-ing-test
  annotations:
    oci-native-ingress.oraclecloud.com/backend-tls-enabled: "false"
    oci-native-ingress.oraclecloud.com/id: <my lb ocid>
    oci-native-ingress.oraclecloud.com/certificate-ocid: <my cert ocid>
    oci-native-ingress.oraclecloud.com/healthcheck-protocol: "HTTP"
    oci-native-ingress.oraclecloud.com/healthcheck-port: "80"
    oci-native-ingress.oraclecloud.com/healthcheck-path: "/"
    oci-native-ingress.oraclecloud.com/healthcheck-return-code: "200"
    oci-native-ingress.oraclecloud.com/healthcheck-force-plaintext: "true"
spec:
  ingressClassName: native-ic-ingress-class
  rules:
    - host: "my-test.hello.io"
      http:
        paths:
          - pathType: Prefix
            path: /app1
            backend:
              service:
                name: web-service
                port:
                  number: 443

apiVersion: v1
kind: Service
metadata:
  name: web-service
  namespace: oci-ing-test
spec:
  type: NodePort
  selector:
    app: web-deployment-v1
  ports:
  - port: 80
    targetPort: 80

[Inbaraj-S]

Update your ingress controller values.yaml to use the latest v1.3.3 release.

[ajhindle]

Thanks, I changed to v1.3.3. I'm getting a repeating error (same as yesterday on v1.3.0) about updating a backend set which doesn't exist (I can't see it in OCI Console either):

util.go:540] Error syncing backends for ingress class native-ic-ingress-class: unable to update backends for native-ic-ingress-class/
bs_e31e79e06990a8d: backendset bs_e31e79e06990a8d was not found

The backend set created by the ingress is actually bs_b20121a6f722474. I don't know whether this error is relevant to my main goal of getting TLS-LB-termination on an existing LB - it's not working yet. In the OCI Console it shows this backend set is created but there is no pod/node attached to it yet (backend state: incomplete).

Do my manifests in the post above look valid?
i.e. using pre-existing LB, port 443 for listener, Port 80 for NodePort service, annotations all good etc ?

[Inbaraj-S]

@ajhindle your service port should be the same as ingress service port 443 Update:

ports:
  - port: 80

to

ports:
  - port: 443

[ajhindle]

Awesome, that worked - thanks @Inbaraj-S . Coming back to the title of this issue, I suggest adding more detailed doco about how to attach ingresses for this controller to existing LBs, and how to do LB-TLS-termination to here and here.
Thanks again!