Add support for IP allowlisting with Upcloud loadbalanacers

[davidumea]

Is your feature request related to a problem? Please describe. We want to be able to support IP allowlisting on ingresses in Upcloud, for this we need to be able to see the client source IP. Upcloud loadbalancers do not understand proxy protocol by default. They have recently added support for this in their loadbalancers, but that config is not exposed in Kubespray.

Describe the solution you'd like Add support for proxy protocol in Upcloud loadbalancers and make sure it works as expected.

Additional context I have created this branch in our Kubespray fork v2.24.0+ipv6-node-local-dns+missing-tf-provider+upcloud-proxy-protocol which includes config for this. (commit https://github.com/elastisys/kubespray/commit/f8c832c8a7d481b2e11b99102e1a62f9128bd508)

With this, setting loadbalancer_proxy_protocol to true in your cluster.tfvars enables proxy protocol in the loadbalancer and in the GUI it gets enabled correctly as expected, however I was not able to get it working with the ingresses I created in the K8s cluster. There may be some other features that needs to be enabled.

With apps installed, you can deploy this "whoami" service in a privileged namespace to check your source IP

Manifests

``` apiVersion: apps/v1 kind: Deployment metadata: name: whoami spec: replicas: 1 selector: matchLabels: app: whoami template: metadata: labels: app: whoami spec: containers: - image: traefik/whoami args: - --port - "10080" name: whoami resources: limits: cpu: 100m memory: 50Mi requests: cpu: 5m memory: 25Mi securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL seccompProfile: type: RuntimeDefault ports: - name: web containerPort: 10080 --- apiVersion: v1 kind: Service metadata: name: whoami spec: ports: - name: web targetPort: 10080 port: 10080 selector: app: whoami --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: cert-manager.io/cluster-issuer: letsencrypt-prod name: whoami-ingress spec: ingressClassName: nginx rules: - host: http: paths: - path: / pathType: Prefix backend: service: name: whoami port: name: web tls: - hosts: - secretName: whoami ```

One thing to note is that proxy protocol needs to be enabled in both the LB frontend and backend:

Definition of done: We can do IP allowlisting on ingresses with Upcloud loadbalancers.

[Xartos]

One thing to note is that proxy protocol needs to be enabled in both the LB frontend and backend:

I interpret that as the opposite, that you need "outbound proxy protocol" to enable "inbound proxy protocol" and not the other way around. But I guess the one who picks this up will need to test it properly

[robinAwallace]

Is there a scenario when we want just outbound proxy protocol? For me it feel more important with inbound, as we use it for ip allow listing.

[Xartos]

Is there a scenario when we want just outbound proxy protocol? For me it feel more important with inbound, as we use it for ip allow listing.

I assumed that "inbound" is equivalent to frontend in haproxy lingo and "outbound" is backend. But maybe I've understood it wrong?

[Xartos]

I took a look at the LBs and the frontend objects on the LB has the "inbound proxy protocol" while the backend objects on the LB has the "outbound proxy protocol" setting.

So I'd say that we will almost always just want to only have the "outbound proxy protocol" set and never the "inbound proxy protocol" because that would mean that the clients that are talking to the LB needs to have proxy protocol as well

[davidumea]

So I'd say that we will almost always just want to only have the "outbound proxy protocol" set and never the "inbound proxy protocol" because that would mean that the clients that are talking to the LB needs to have proxy protocol as well

Might explain this :smile:

however I was not able to get it working with the ingresses I created in the K8s cluster. There may be some other features that needs to be enabled.

[davidumea]

I took a look at the LBs and the frontend objects on the LB has the "inbound proxy protocol" while the backend objects on the LB has the "outbound proxy protocol" setting.

So I'd say that we will almost always just want to only have the "outbound proxy protocol" set and never the "inbound proxy protocol" because that would mean that the clients that are talking to the LB needs to have proxy protocol as well

Yep it works as expected with only outbound proxy protocol :facepalm:, thanks :smile:

[davidumea]

I created a ck8s-kubespray branch v2.24.0-ck8s1+upcloud-lb-proxy-protocol which includes a commit for adding proxy protocol support in Upclouds LB, this now works as expected thanks to @Xartos input :slightly_smiling_face:

I also included that commit in the upcoming Kubespray release/patch we're doing https://github.com/elastisys/ck8s-issue-tracker/issues/182

I will also upstream this.