aws / aws-sdk-java-v2

The official AWS SDK for Java - Version 2
Apache License 2.0
2.2k stars 848 forks source link

How to configure a proxy without SSL #1234

Closed bsannicolas closed 4 years ago

bsannicolas commented 5 years ago

Hey, I'm trying to set up the ApacheHttpClient to use a proxy without SSL, but I haven't been able to figure it out. I don't want SSL because my service makes all requests to localhost and a local proxy handles SSL with all remote services.

Here's my proxy configuration

    val proxyEndpoint = URI.create(s"http://${proxyConfig.host}:${proxyConfig.port}")
    val awsProxyConfig = ProxyConfiguration
      .builder()
      .username(proxyConfig.username)
      .password(proxyConfig.password)
      .endpoint(proxyEndpoint)
      .build()
    val awsHttpClient = ApacheHttpClient
      .builder()
      .withProxyConfiguration(awsProxyConfig)
      .build()

I set the scheme to "http" hoping to get an unencrypted connection based on this:

        private Registry<ConnectionSocketFactory> createSocketFactoryRegistry(ConnectionSocketFactory sslSocketFactory) {
            return RegistryBuilder.<ConnectionSocketFactory>create()
                    .register("http", PlainConnectionSocketFactory.getSocketFactory())
                    .register("https", sslSocketFactory)
                    .build();
        }

but I get trust store errors (I don't have a trust store set up) with a stack trace through SSLConnectionSocketFactory.

Caused by: javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946)
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1903)
        at sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1886)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1402)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
        at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396)
        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.upgrade(DefaultHttpClientConnectionOperator.java:193)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.upgrade(PoolingHttpClientConnectionManager.java:389)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at software.amazon.awssdk.http.apache.internal.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:80)
        at com.sun.proxy.$Proxy2.upgrade(Unknown Source)
        at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:429)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
        at software.amazon.awssdk.http.apache.internal.impl.ApacheSdkHttpClient.execute(ApacheSdkHttpClient.java:72)
        at software.amazon.awssdk.http.apache.ApacheHttpClient.execute(ApacheHttpClient.java:233)
        at software.amazon.awssdk.http.apache.ApacheHttpClient.access$500(ApacheHttpClient.java:102)
        at software.amazon.awssdk.http.apache.ApacheHttpClient$1.call(ApacheHttpClient.java:214)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.executeHttpRequest(MakeHttpRequestStage.java:66)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.execute(MakeHttpRequestStage.java:51)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.execute(MakeHttpRequestStage.java:35)
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:64)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:36)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:77)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:39)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage$RetryExecutor.doExecute(RetryableStage.java:113)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage$RetryExecutor.execute(RetryableStage.java:86)
        ... 28 common frames omitted

I also tried using a route planner and credentials provider directly instead of the proxy configuration:

    val awsHttpClient = ApacheHttpClient
      .builder()
      .httpRoutePlanner(new SdkProxyRoutePlanner(proxyConfig.host, proxyConfig.port, "http", Set.empty[String].asJava))
      .credentialsProvider(ApacheUtils.newProxyCredentialsProvider(awsProxyConfig))
      .build()

This also doesn't work. However, if I leave off the credentials provider it will in fact use the PlainConnectionSocketFactory (of course, I get 407 Proxy Authentication Required, so I need the credentials). Is there a way to configure this?

dagnir commented 5 years ago

Hi @bsannicolas, you shouldn't need to use a customs socket factory registry or route planner. Have you tried just with your first code snippet, without the custom factory registry or route planner?

    val proxyEndpoint = URI.create(s"http://${proxyConfig.host}:${proxyConfig.port}")
    val awsProxyConfig = ProxyConfiguration
      .builder()
      .username(proxyConfig.username)
      .password(proxyConfig.password)
      .endpoint(proxyEndpoint)
      .build()
    val awsHttpClient = ApacheHttpClient
      .builder()
      .withProxyConfiguration(awsProxyConfig)
      .build()

A quick local test with the following code shows the SDK using plaintext HTTP to the proxy:

        S3Client s3 = S3Client.builder()
                .httpClient(ApacheHttpClient.builder()
                        .proxyConfiguration(ProxyConfiguration.builder()
                                .endpoint(HTTP_PROXY_ENDPOINT)
                                .build())
                        .build())
                .build();

        s3.listBuckets();
bsannicolas commented 5 years ago

The problem is that it doesn't work when credentials are added. Internally, it looks like adding proxy configuration is the same as adding an httpRoutePlanner and, if authenticated, a credentials provider:

    private void addProxyConfig(HttpClientBuilder builder,
                                DefaultBuilder configuration) {
        ProxyConfiguration proxyConfiguration = configuration.proxyConfiguration;

        Validate.isTrue(configuration.httpRoutePlanner == null || !isProxyEnabled(proxyConfiguration),
                        "The httpRoutePlanner and proxyConfiguration can't both be configured.");
        Validate.isTrue(configuration.credentialsProvider == null || !isAuthenticatedProxy(proxyConfiguration),
                        "The credentialsProvider and proxyConfiguration username/password can't both be configured.");

        HttpRoutePlanner routePlanner = configuration.httpRoutePlanner;
        if (isProxyEnabled(proxyConfiguration)) {
            log.debug(() -> "Configuring Proxy. Proxy Host: " + proxyConfiguration.host());
            routePlanner = new SdkProxyRoutePlanner(proxyConfiguration.host(),
                                                    proxyConfiguration.port(),
                                                    proxyConfiguration.scheme(),
                                                    proxyConfiguration.nonProxyHosts());
        }

        CredentialsProvider credentialsProvider = configuration.credentialsProvider;
        if (isAuthenticatedProxy(proxyConfiguration)) {
            credentialsProvider = ApacheUtils.newProxyCredentialsProvider(proxyConfiguration);
        }

        if (routePlanner != null) {
            builder.setRoutePlanner(routePlanner);
        }

        if (credentialsProvider != null) {
            builder.setDefaultCredentialsProvider(credentialsProvider);
        }
    }

I've found that using only the http route planner does create an plaintext socket but if you specify a credentials provider this is no longer the case.

debora-ito commented 4 years ago

@bsannicolas apologies for the super long silence here. Are you still experiencing this issue? If so please reply with a comment and I'll investigate.

bsannicolas commented 4 years ago

I think this ended up being related to our proxy configuration, but I can't say I remember the exact details. I don't think there is a problem with the SDK.

On Wed, Oct 7, 2020 at 8:28 PM Debora N. Ito notifications@github.com wrote:

@bsannicolas https://github.com/bsannicolas apologies for the super long silence here. Are you still experiencing this issue? If so please reply with a comment and I'll investigate.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/aws/aws-sdk-java-v2/issues/1234#issuecomment-705261947, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL5XM2ZKYWQVYSFUQLYWNBTSJUBRBANCNFSM4HJHBUYA .

debora-ito commented 4 years ago

Thank you for the follow-up. Feel free to reach out if you have further issues.