Allow using an upstream HTTP proxy in forward signing mode

[rassie]

Summary

Currently, in forward signing mode an HTTP proxy can be set for gatekeeper to use (e.g. using the standard UNIX environment variables http_proxy / https_proxy / no_proxy). However, it will only be used to communicate with the IdP (e.g. Keycloak) for authentication and token fetching. Any request going through gatekeeper won't be using said HTTP proxy, which in many cases prevents usage of gatekeeper in environments where such proxies are mandatory.

How

In the simplest form, the environment variables http_proxy / https_proxy / no_proxy should be used to configure upstream communication. Additional command line switches, configuration files etc. are welcome, but not necessary for this feature to get to a working state. Separate proxies for IdP and upstream communication and/or rules and routing could become interesting additions in the future.

Acceptance criteria

A gatekeeper instance configured with an HTTP proxy should use that proxy both for IdP and upstream communication (except for no_proxy rules). This could be checked with a proxy like mitmproxy, which would display the requests in detail.

[p53]

@rassie could you try 2.10.0-rc1 pre-release, there is also docker image, haven't added no_proxy yet, added option --upstream-proxy

[rassie]

Thanks for quick implementation. It seems to work fine, i.e. I could access a restricted web page with authorization. Would love it if you added no_proxy support, makes many things easier! Is it possible to re-use whatever library you are using for the providers? It seems to automatically use the environment variables and probably parses a lot of edge cases with no_proxy.

[p53]

@rassie hmm i cannot reuse same library but i could maybe pass it from provider client to upstream settings, but what if we would like to have separate no_proxy for providers and upstream, will think about it, right that no_proxy is quite non-standardised it would be probably best not to re-implent its working

[p53]

@rassie i used golang net library function, altough it has some issues which affect testing, can you try? Use HTTPS_PROXY/HTTP_PROXY/NO_PROXY for Idp provider proxies and for upstream proxy use UPSTREAM_PROXY/UPSTREAM_NO_PROXY - 2.10.0-rc2

[rassie]

I'll be testing today, will need to create a small testbench for it though -- somehow I can't put multiple domains in --forwarding-domains right now and that's a small roadblock.

[p53]

@rassie you are defining it like this?:

  - --forwarding-domains=projecta.svc.cluster.local
  - --forwarding-domains=projectb.svc.cluster.local

[rassie]

Yes, I do and still suspect it doesn't work. Still a suspicion though, will give you proof when I have it. Somehow I stumble from one problem to another. ~For example, --skip-upstream-tls-verify=true does what it says on the packaging, i.e. it does not skip TLS verification for the IdP (I prefer not doing too much on live staging servers, so I'm testing locally with mitmproxy).~ Will hopefully have something tangible by tomorrow.

EDIT: Found --skip-openid-provider-tls-verify.

[rassie]

OK, it seems using mitmproxy is a bad idea, since it's stripping auth headers. Will find a better solution.

[rassie]

So, I hope my results are more or less correct:

• --forwarding-domains works fine with multiple values, the problem was in my test bench
• --upstream-proxy and --upstream-no-proxy work correctly
• UPSTREAM_PROXY and UPSTREAM_NO_PROXY do not work, correct variable names are with a prefix: PROXY_UPSTREAM_PROXY and PROXY_UPSTREAM_NO_PROXY.

So all in all: yes, it works, thanks! Time to battle-test this!

[p53]

@rassie yes every env variable, which has cli flag needs PROXY_ prefix (as specified also in configuration reference), sorry i am used to it so I didn't realize I need to write it...https://gogatekeeper.github.io/gatekeeper/configuration/