Enable local gateway mode on nerc-ocp-prod cluster

[larsks]

In order to properly route connections to the NESE cluster, we need to enable local gateway mode in our OVNKubernetes configuration 1. We've already done this on the infra cluster so in theory we just need to include the right resource in the nerc-ocp-prod overlay.

[naved001]

@larsks I could use a second set of eyes on this

On nerc-ocp-prod, I see that networkType is OpenShiftSDN.

naved@Naveds-MacBook-Pro ~ % oc get network cluster -o yaml
apiVersion: config.openshift.io/v1
kind: Network
metadata:
  creationTimestamp: "2022-08-17T17:05:53Z"
  generation: 2
  name: cluster
  resourceVersion: "3571"
  uid: 3395c8fe-f370-421d-a0f8-40ccd93fd0b9
spec:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  externalIP:
    policy: {}
  networkType: OpenShiftSDN
  serviceNetwork:
  - 172.30.0.0/16
status:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  clusterNetworkMTU: 1450
  networkType: OpenShiftSDN
  serviceNetwork:
  - 172.30.0.0/16

Also, why is the apiVersion config.openshift.io/v1, the documents have the apiVersion operator.openshift.io?

On nerc-ocp-infra I don't see the gateway setting

naved@Naveds-MacBook-Pro ~ % oc get network cluster -o yaml
apiVersion: config.openshift.io/v1
kind: Network
metadata:
  creationTimestamp: "2022-05-26T15:12:38Z"
  generation: 2
  name: cluster
  resourceVersion: "4395"
  uid: f0ba9565-ebb8-457d-9392-28b53c26d578
spec:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  externalIP:
    policy: {}
  networkType: OVNKubernetes
  serviceNetwork:
  - 172.30.0.0/16
status:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  clusterNetworkMTU: 1400
  networkType: OVNKubernetes
  serviceNetwork:
  - 172.30.0.0/16

Am I not looking at the right place?

[larsks]

On nerc-ocp-prod, I see that networkType is OpenShiftSDN.

Arg, it shouldn't be. I think at some point the cluster was rebuilt and that's why we're running 4.11 instead of 4.10 and also explains the SDN difference. We'll have to add that to "things to fix when we rebuild the cluster", but the good news is that when using openshiftsdn we don't have to work around any routing issues.

[naved001]

but the good news is that when using openshiftsdn we don't have to work around any routing issues.

I guess I should undo the change then and move the OVNkubernetes spec back into the overlay for nerc-ocp-infra

[naved001]

Arg, it shouldn't be. We'll have to add that to "things to fix when we rebuild the cluster"

I am a bit confused about what the plan is going forward, do we want to use OpenShiftSDN which wouldn't require the workaround or not? Or will we rebuild it and run 4.10 with OVNKubernetes which would require the workaround?

[larsks]

The latter (rebuild with 4.10 and ovnkubernetes).

[naved001]

okay, in that case I will let the change be there which should be in effect after the rebuild then.

[naved001]

On nerc-ocp-infra I don't see the gateway setting

I need to be looking at network.operator.openshift.io instead of network.config.openshift.io

{
  "ovnKubernetesConfig": {
    "gatewayConfig": {
      "routingViaHost": true
    },
    "genevePort": 6081,
    "mtu": 1400,
    "policyAuditConfig": {
      "destination": "null",
      "maxFileSize": 50,
      "rateLimit": 20,
      "syslogFacility": "local0"
    }
  },
  "type": "OVNKubernetes"
}