search

digitalocean / Kubernetes-Starter-Kit-Developers

Hands-on tutorial and Automation stack for an operations-ready DigitalOcean Kubernetes (DOKS) cluster.
745 stars 258 forks source link

[Proxy Protocol] Fix proxy protocol issue from the Ingress Controller chapter #91

Closed vladciobancai closed 2 years ago

vladciobancai commented 2 years ago

Problem Description

By default the protocol used by the DigitalOcean Load Balancers is tcp and when the proxy protocol is enabled the requests fails to get to backend droplets and the flag do-loadbalancer-tls-passthrough needs to be enabled The following fix PR https://github.com/digitalocean/Kubernetes-Starter-Kit-Developers/pull/90 will fix the problem partially.

in the Ingress Controller chapter the following annotation page should be added Service Annotations to allow users to use other services when the proxy protocol is configured

Impacted Areas

Setup Ingress Controller chapter.

Prerequisites

N/A.

Steps to Reproduce

- Nginx

  1. Create / Setup the Ingress Controller from steps 01-05 Ingress Controller Nginx
  2. Validate the Ingress Controller 
    
Request served by echo-5d8d65c665-fpbwx

HTTP/1.1 GET /

Host: .... X-Forwarded-Scheme: https X-Scheme: https User-Agent: curl/7.77.0 X-Request-Id: 6b066deeed56ee989b49269d5dce24b5 X-Real-Ip: 10.110.0.3 X-Forwarded-Host: echo.vlad.bond0.site X-Forwarded-Port: 443 X-Forwarded-Proto: https X-Forwarded-For: 10.110.0.3

3. Run the following steps [Proxy Protocol](https://github.com/digitalocean/Kubernetes-Starter-Kit-Developers/blob/main/03-setup-ingress-controller/nginx.md#step-6---enabling-proxy-protocol) to enable it
4. The validation will fail with the following erors
400 Bad Request

400 Bad Request

nginx
5. At the current stage the Ingress Controller is broken and even with a helm rollback to a previous revision will not fix it. 

#### - Ambassador

1. Create / Setup the Ingress Controller from steps 01-05 [Ingress Controller Ambassador](https://github.com/digitalocean/Kubernetes-Starter-Kit-Developers/blob/main/03-setup-ingress-controller/ambassador.md)

2. Validate the Ingress Controller

HTTP/1.1 200 OK content-type: text/plain date: Wed, 22 Dec 2021 08:53:28 GMT content-length: 356 x-envoy-upstream-service-time: 0 server: envoy

Request served by echo-5d8d65c665-8spcr

HTTP/1.1 GET /

Host: .... X-Forwarded-For: 79.119.116.72 X-Forwarded-Proto: https X-Envoy-Original-Path: /echo/ User-Agent: curl/7.77.0 Accept: / X-Envoy-External-Address: 79.119.116.72 X-Request-Id: a3a148ea-3dee-4596-878e-d924b54be45f X-Envoy-Expected-Rq-Timeout-Ms: 3000 Content-Length: 0

3. Run the following steps [Proxy Protocol](https://github.com/digitalocean/Kubernetes-Starter-Kit-Developers/blob/main/03-setup-ingress-controller/ambassador.md#step-6---enabling-proxy-protocol) to enable it

4. The validation will fail with the following erors

curl: (52) Empty reply from server curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to :443


### Expected Results
The Ingress Controller should balance the traffic in both scenarios

### Proposal
#### - Nginx

The following annotation `service.beta.kubernetes.io/do-loadbalancer-tls-passthrough: true` should be adding the helm value file [03-setup-ingress-controller/assets/manifests/nginx-values-v4.0.6.yaml](https://github.com/digitalocean/Kubernetes-Starter-Kit-Developers/blob/main/03-setup-ingress-controller/assets/manifests/nginx-values-v4.0.6.yaml)

service: type: LoadBalancer annotations:

Enable proxy protocol

  service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: "true"
  service.beta.kubernetes.io/do-loadbalancer-tls-passthrough: "true"

config: use-proxy-protocol: "true"

Will enable the proxy protocol in the Ingress Controller Nginx config on the pod(s)

#### - Ambassador

The following annotation should be added/enabled in `03-setup-ingress-controller/assets/manifests/ambassador-values-v6.7.13.yaml`

service: type: LoadBalancer annotations:

You can keep your existing LB when migrating to a new DOKS cluster, or when reinstalling AES

kubernetes.digitalocean.com/load-balancer-id: ""

service.kubernetes.io/do-loadbalancer-disown: false

  service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: "true"
  service.beta.kubernetes.io/do-loadbalancer-tls-passthrough: "true"
Update the configuration running helm upgrade

HELM_CHART_VERSION="6.7.13"

helm upgrade ambassador datawire/ambassador --version "$HELM_CHART_VERSION" \ --namespace ambassador -f "03-setup-ingress-controller/assets/manifests/ambassador-values-v${HELM_CHART_VERSION}.yaml"


Enable the Proxy Protocol for Ambassador using the steps described https://github.com/digitalocean/Kubernetes-Starter-Kit-Developers/blob/main/03-setup-ingress-controller/ambassador.md#step-6---enabling-proxy-protocol