[75] add http check for ingress

[axisofentropy]

What we really want is for our redirect page to redirect when or just after when our ingress-nginx load balancer begins forwarding traffic, i.e. when HTTP requests to the load balancer will successfully be forwarded to the tenant's container. When does ingress-nginx make that determination? Can we check the status of the Ingress or containers and achieve this without making our own HTTP requests?

[zipofar]

I think ingress starts forward traffic immediately after we add ingress rule in the namespace. So I guess we can't avoid make http checks. But maybe we can separate ingress-nginx response from tenant's container. Maybe we can add custom header to our ingress-nginx and parse it. Or if that's impossible, we can add custom text to the error body page and parse it. What do you think?

Regarding 500 error. I thought we can't get status lower then 500 from out ingress-nginx because before http check we make tcp checks, therefore 404 and other statuses are impossible in this case. But I forgot that we can have 401 if user use basic auth feature.

How it works now. We make two tcp checks. One for service (to internal ip address like a 10.20.0.7) with public port and one for ingress address (like a pr-4-deployment-168076242045401-hello-world-2.app.qa-gke.uffizzi.com). Then if the previous tcp checks have done we make http check to https://pr-4-deployment-168076242045401-hello-world-2.app.qa-gke.uffizzi.com

[axisofentropy]

Thanks for explaining all that. It's good that you're checking at the load balancer, that serves our goal.

I think nginx itself, within its controller, maintains its own status of each backend, but I don't think that's exposed in a way we can easily access?

You're right that we want to determine whether an error response is coming from our load balancer or the tenant's container. We recently added custom error responses. I think they have some specific HTTP response headers we can check. That will be better than checking the response body, which could change.

[axisofentropy]

Well, it looks like there are no such response headers. Maybe we can add them?

[zipofar]

Looks like we can add custom headers to ingress (doc)

[zipofar]

I've added ConfigMap ingress-nginx-controller-custom-headers.yml

apiVersion: v1
data:
  X-Different-Uffizzi-Name: "true"
  X-Request-Uffizzi-Start: t=${msec}
  X-Using-Nginx-Uffizzi-Controller: "true"
kind: ConfigMap
metadata:
  name: ingress-nginx-controller-custom-headers
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

And have updated ConfigMap ingress-nginx-for-uffizzi-controller-controller

apiVersion: v1
data:
  allow-snippet-annotations: "true"
  custom-http-errors: 400,401,403,404,502,503,504
  disable-proxy-intercept-errors: "true"
  proxy-body-size: 100m
  proxy-buffer-size: 128k
  add-headers: "ingress-nginx/ingress-nginx-controller-custom-headers" <<<<<<<<<
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: ingress-nginx-for-uffizzi-controller
    meta.helm.sh/release-namespace: ingress-nginx
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx-for-uffizzi-controller
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.7.0
    helm.sh/chart: ingress-nginx-4.6.0
  name: ingress-nginx-for-uffizzi-controller-controller
  namespace: ingress-nginx

And it works


[zipofar]

But ingress return these headers always.


Therefore separate response by custom headers does not work

[axisofentropy]

Yeah we want to add them to the "default backend" specifically