Interop with other Ingress Controllers

[wasaga]

Is your feature request related to a problem? Please describe.

Pomerium and Pomerium Ingress Controller provide a feature rich ingress controller with built-in authentication, authorization and audit capabilities. Pomerium uses Envoy as its reverse proxy component and may be relied on for many demanding workflows and configuration options.

There could be situations when it is desirable to run Pomerium behind an existing Ingress Controller, i.e.

• utilize an existing organization-wide traffic management, monitoring and alerting.
• receive certain workloads Pomerium is not specialized to handle - i.e. API gateway.
• perform request transformations, routing and scripting, configure rate limiting, etc.
• use 3rd party plugins such as WAF, anti-DDOS, etc.

Historically, Pomerium supports forward-auth mode for NGINX and Traefik that allows you to authenticate and authorize requests using Pomerium. However, there are dozens of Ingress Controllers on the market, including those natively provided by the cloud vendors and integrating with each and every of them does not sound feasible.

Describe the solution you'd like

Suppose you have a typical Ingress resource that is talking to your app Service, and you would like to add a drop-in authentication policy that would only allow users from your domain to access the application.

Pomerium Ingress Controller would additionally scan Ingress resources with ingressClass other than pomerium that have ingress.pomerium.io annotations and and create a proxying Service that would implement authentication and authorization.

Thus if you kubectl apply the below spec, Pomerium Controller mutation webhook would

1. Create a pomerium-my-app Service, pointing to Pomerium Proxy.
2. Configure Pomerium Proxy Route to your original Service my-app, including applying all the rules set via pomerium-specific Ingress annotations.
3. mutate the incoming Ingress resource so that my-app would be replaced with pomerium-my-app Service.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minimal-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    ingress.pomerium.io/policy: |
      - allow:
          and:
            - domain:
                is: pomerium.com
spec:
  ingressClassName: nginx-example
  rules:
  - http:
      paths:
      - path: /testpath
        pathType: Prefix
        backend:
          service:
            name: my-app
            port:
              number: 80

Describe alternatives you've considered

Explain any additional use-cases

Additional context

[travisgroth]

I think we could skip (2) and have step (3) insert the pomerium-proxy service rather than creating a new one for each ingress. I'm not sure there's any practical difference and would reduce the number of moving parts.

[wasaga]

how would proxy know which host it is called for? the Host would always be http://pomerium-proxy in that case.

[travisgroth]

I was thinking the external proxy would be responsible for ensuring the incoming host header is retained and we match directly on whatever is in the ingress rules. However, in thinking about it, that might cause issues with routing if using TLS.

That said, we still might have an issue with TLS matching if we provision individual services too. I'm not familiar enough with the listener config/behavior to be certain without trying it. I believe the way TLS and routing are related in envoy has some sharp edges we'll need to be mindful of in either solution.

[wasaga]

related: https://github.com/pomerium/ingress-controller/issues/177

[desimone]

Wildcard routing related:

• https://github.com/pomerium/pomerium/issues/659
• https://github.com/pomerium/pomerium/issues/1677

[iakat]

Considering the removal of forward auth at v0.21.0 - this seems to be an issue which will not be solved/addressed.

It is a shame, as Pomerium+fwdauth hit the sweet spot between doing one thing right and playing well with other services. Unfortunately will have to deploy an alternative as I require features ingress-nginx can provide, but Pomerium cannot.

[wasaga]

I require features ingress-nginx can provide, but Pomerium cannot.

it would be of great help if you could help us identify the missing features.

[iakat]

The missing feature does not matter! Removing this feature locks you in using a Pomerium instead of any perfectly good ingress controllers, which have been battle-tested in production for decades, with all the features and community supports that come with it.

But I'll humor you: running custom Lua (openresty), rate limiting, header manipulation, proxy protocol, scenarios where the upstream ingress loadbalancer is already being exposed and exposing more IPs is not viable, .......

[iakat]

In addition, working well with other proxies can help mitigate critical CVEs.