Setting http proxy into OAuth client connection

[hplscosta]

After defining proxy.host and proxy.port into Kafka Producer Config, I'm getting a connection error obtaining JWT token:

I didn't find a way to add proxy config into the HttpURLConnection generated on io.strimzi.kafka.oauth.common.HttpUtil.

Setting proxy config into JVM will not solve my problem as I need to call internal services without proxy.

Logs:

java.io.IOException: Failed to connect to: https://keycloak/auth/realms/master/protocol/openid-connect/token
at io.strimzi.kafka.oauth.common.HttpUtil.request(HttpUtil.java:120) ~[kafka-oauth-common-0.5.0.jar!/:?]
at io.strimzi.kafka.oauth.common.HttpUtil.post(HttpUtil.java:62) ~[kafka-oauth-common-0.5.0.jar!/:?]
at io.strimzi.kafka.oauth.common.OAuthAuthenticator.post(OAuthAuthenticator.java:92) ~[kafka-oauth-common-0.5.0.jar!/:?]
(...)
Caused by: java.net.ConnectException: Connection timed out (Connection timed out)
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:?]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java

[mstruk]

We delegate HTTP calls to java.net.HttpURLConnection. That one does not support setting proxying per host. But you can set non-proxy hosts. See: https://docs.oracle.com/javase/7/docs/api/java/net/doc-files/net-properties.html#Proxies

You have a very niche case here. Why do you need this?

[hplscosta]

Thanks @mstruk !

We have a Kafka cluster on AWS and producers running on-prem. Clients must use proxy for external communications, it's an environment requirement.

On io.strimzi.kafka.oauth.common.HttpUtil the connection is open like this:

con = (HttpURLConnection) uri.toURL().openConnection();

Why not set proxy when exists? Like this:

con = (HttpURLConnection) uri.toURL().openConnection(new Proxy());

[mstruk]

When client proxies are used, the client C has to connect to proxy P rather than to authorization server S. In order to do this securely with TLS connection the TLS termination is performed by proxy P rather than the server S. Client can be sure of integrity between C and P, but not between P and S. That's a big problem as you should treat all communication between C and S as confidential, otherwise it's open to leak your secrets. Can you trust P with your client secret for example? Can you trust it doesn't log the traffic and one day lose log files? Can you trust that communication between P and S is properly secured with TLS? To me using a client proxy between client and authorization server sounds like a problematic system design.

[hplscosta]

I agree with your reasons but of course proxy is optionally configured.

In this case, all requests must go through our proxy (that we trust), including kafka client and schema registry client, both already configured to use proxy.

[cc12345678]

We have the similar design which OAuthBearer will need to go thru our proxy(basic auth), would there be an enhancement to accept the proxy with authentication?

[mstruk]

@cc12345678 How will you get your Kafka client to use your HTTP proxy? Kafka uses a custom protocol over the direct TCP connection - it does not speak HTTP, and has no client proxy configuration support. As far as I can see, your only option is to route traffic at network level. If you can't use HTTP proxy for the Kafka traffic, then using it for OAuth authentication is irrelevant.

[cc12345678]

@mstruk previously we using HTTP proxy WITHOUT BASIC AUTHENTICATION with our same codebase everything work perfect. kafka config SSL_TRUSTSTORE_LOCATION SSL_TRUSTSTORE_PASSWORD SSL_TRUSTSTORE_TYPE SECURITY_PROTOCOL to SASL_SSL, SASL_MECHANISM to OAUTHBEARER, OAuthBearerLoginModule::class.java.name required OAUTH_CLIENT_ID= OAUTH_CLIENT_SECRET= OAUTH_TOKEN_ENDPOINT_URI= A

However, recently we did some migration of our infrastructure this include to use a new HTTP proxy with basic authentication. it lead to the following error during connection to A

"Remote host terminated the handshake","logger_name":"org.apache.kafka.common.security.oauthbearer. ││ OAuthBearerLoginModule","thread_name":"task-1","level":"ERROR","level_value":40000,"stack_trace":"javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake\n\tat java.base/sun.security.ssl.S ││ SLSocketImpl.handleEOF(Unknown Source)\n\tat java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source)\n\tat java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source)\n\tat java.bas ││ e/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)\n\tat java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)\n\tat java.base/sun.net.www.protocol.https.HttpsClient.afterConn ││ ect(Unknown Source)\n\tat java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)\n\tat java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(Unknown So ││ urce)\n\tat io.strimzi.kafka.oauth.common.HttpUtil.request(HttpUtil.java:131)\n\tat io.strimzi.kafka.oauth.common.HttpUtil.request(HttpUtil.java:85)\n\tat io.strimzi.kafka.oauth.common.HttpUtil.post(HttpUti ││ l.java:61)\n\tat io.strimzi.kafka.oauth.common.OAuthAuthenticator.post(OAuthAuthenticator.java:116)\n\tat io.strimzi.kafka.oauth.common.OAuthAuthenticator.loginWithClientSecret(OAuthAuthenticator.java:72)\n

[mstruk]

@cc12345678 It looks like the endpoint url you have configured is expected to provide TLS but the fact that connection was closed suggests that it probably expects plaintext HTTP request. This might be a result of some network level interception / redirection of requests.

But for any discussion around proxy support you would really have to describe your network system design in detail. Your solution no. 1 is to get an exception for your hosts and ports since as I said Kafka client to Kafka broker communication can not be proxied over HTTP proxies. The only debatable case then is really if all your Kafka clients and Kafka brokers can speak directly, but only connection to your authorization server needs to go over an HTTP proxy. If that is your scenario I would like to better understand the rationale behind such a situation. Also the easiest solution would be to make an exception and allow direct connections to the specific authorization server. Another option is to set up a reverse proxy server with ssl passthrough in front of your authorization server and allow requests through that. It creates a transparent tunnel so that the Kafka client isn't even aware of it. Such a solution is not problematic as the client still establishes a secure connection directly with the destination authorization server.

[cc12345678]

@mstruk Our kafka broker communication is running no proxy. However, A is an internet endpoint url. In order to access A, it have to go thru proxy (no auth) which now it have changed to proxy with "basic auth"

On io.strimzi.kafka.oauth.common.HttpUtil the connection is open as this:

con = (HttpURLConnection) uri.toURL().openConnection();

if it to enhance to proxy authentication the issue will be fixed.

[mstruk]

@cc12345678 How did you configure your client to connect to A through a proxy (no auth)?

I suppose you used http.proxyHost and http.proxyPort system properties. In that case you might try set the following additional system properties: http.proxyUser, http.proxyPassword, and set jdk.http.auth.tunneling.disabledSchemes to empty string ("-Djdk.http.auth.tunneling.disabledSchemes=").

[mstruk]

About the last system property above - it will enable Basic authentication over unsecured HTTP connection to the proxy, but that is clearly unsafe. You should not use that for any kind of serious setup. Rather your HTTP proxy should use TLS, and you should use https.proxyHost and https.proxyPort in your client configuration.

[cc12345678]

@mstruk thank for your feedback and suggestion