Customize the http client builder

[juriohacc]

Hello,

As i have already mentioned the need to configure http client connection TTL for spring cloud vault (https://github.com/spring-cloud/spring-cloud-vault/issues/660), i need to find the good way to do it.

You gave us a way to do it : https://gist.github.com/mp911de/157e6ae14ba6bb3565c36b425d3d83b7 However, even if the class HttpComponents become public, there is no method getBuilder()

Here is the static class HttpComponents in ClientHttpRequestFactoryFactory :

    static class HttpComponents {
        HttpComponents() {
        }

        static ClientHttpRequestFactory usingHttpComponents(ClientOptions options, SslConfiguration sslConfiguration) throws GeneralSecurityException, IOException {
            HttpClientBuilder httpClientBuilder = HttpClients.custom();
            httpClientBuilder.setRoutePlanner(new SystemDefaultRoutePlanner(DefaultSchemePortResolver.INSTANCE, ProxySelector.getDefault()));
            if (ClientHttpRequestFactoryFactory.hasSslConfiguration(sslConfiguration)) {
                SSLContext sslContext = ClientHttpRequestFactoryFactory.getSSLContext(sslConfiguration, ClientHttpRequestFactoryFactory.getTrustManagers(sslConfiguration));
                String[] enabledProtocols = null;
                if (!sslConfiguration.getEnabledProtocols().isEmpty()) {
                    enabledProtocols = (String[])sslConfiguration.getEnabledProtocols().toArray(new String[0]);
                }

                String[] enabledCipherSuites = null;
                if (!sslConfiguration.getEnabledCipherSuites().isEmpty()) {
                    enabledCipherSuites = (String[])sslConfiguration.getEnabledCipherSuites().toArray(new String[0]);
                }

                SSLConnectionSocketFactory sslSocketFactory = new SSLConnectionSocketFactory(sslContext, enabledProtocols, enabledCipherSuites, SSLConnectionSocketFactory.getDefaultHostnameVerifier());
                httpClientBuilder.setSSLSocketFactory(sslSocketFactory);
                httpClientBuilder.setSSLContext(sslContext);
            }

            RequestConfig requestConfig = RequestConfig.custom().setConnectTimeout(Math.toIntExact(options.getConnectionTimeout().toMillis())).setSocketTimeout(Math.toIntExact(options.getReadTimeout().toMillis())).setAuthenticationEnabled(true).build();
            httpClientBuilder.setDefaultRequestConfig(requestConfig);
            httpClientBuilder.setRedirectStrategy(new LaxRedirectStrategy());
            return new HttpComponentsClientHttpRequestFactory(httpClientBuilder.build());
        }
    }

The only method available from this static class HttpComponents is usingHttpComponents but this method need some parameters (clientOptions and SslConfiguration) and return ClientHttpRequestFactory This method does not let configure the HttpClientBuilder and from Spring Boot main method. How could we get the clientOptions and SslConfiguration which necessitate to inject VaultProperties.

This is a code i have done, do you have a better way before wait for the change apply in new spring vault core version about visibility of HttpComponents ?

Here is the code i have done :

public static void main(String[] args) throws GeneralSecurityException, IOException {

        SpringApplication app = new SpringApplication(StandardsMicroserviceApplicationTest.class);

        // how could we get it ? We need to retreive data from VaultProperties to instanciate these classes
        ClientOptions clientOptions = null;
        SslConfiguration sslConfiguration = null;

        // create http client builder from copied code method usingHttpComponents from ClientHttpRequestFactoryFactory
        HttpClientBuilder builder = customUsingHttpComponents(clientOptions, sslConfiguration);

        HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory(builder.build());

        app.addBootstrapRegistryInitializer(registry -> {
            registry.register(AbstractVaultConfiguration.ClientFactoryWrapper.class,
                    BootstrapRegistry.InstanceSupplier.of(new AbstractVaultConfiguration.ClientFactoryWrapper(requestFactory)));
        });

        app.run(args);
        log.info("Application started");
    }

    // copied code from ClientHttpRequestFactoryFactory class and change it to return HttpClientBuilder instead
    public static HttpClientBuilder customUsingHttpComponents(ClientOptions options, SslConfiguration sslConfiguration) throws GeneralSecurityException, IOException {
        HttpClientBuilder httpClientBuilder = HttpClients.custom();

        //customize http client builder connection TimeToLive
        httpClientBuilder = httpClientBuilder.setConnectionTimeToLive(120, TimeUnit.SECONDS);

        httpClientBuilder.setRoutePlanner(new SystemDefaultRoutePlanner(DefaultSchemePortResolver.INSTANCE, ProxySelector.getDefault()));
        if (ClientHttpRequestFactoryFactory.hasSslConfiguration(sslConfiguration)) {
            SSLContext sslContext = ClientHttpRequestFactoryFactory.getSSLContext(sslConfiguration, ClientHttpRequestFactoryFactory.getTrustManagers(sslConfiguration));
            String[] enabledProtocols = null;
            if (!sslConfiguration.getEnabledProtocols().isEmpty()) {
                enabledProtocols = (String[])sslConfiguration.getEnabledProtocols().toArray(new String[0]);
            }

            String[] enabledCipherSuites = null;
            if (!sslConfiguration.getEnabledCipherSuites().isEmpty()) {
                enabledCipherSuites = (String[])sslConfiguration.getEnabledCipherSuites().toArray(new String[0]);
            }

            SSLConnectionSocketFactory sslSocketFactory = new SSLConnectionSocketFactory(sslContext, enabledProtocols, enabledCipherSuites, SSLConnectionSocketFactory.getDefaultHostnameVerifier());
            httpClientBuilder.setSSLSocketFactory(sslSocketFactory);
            httpClientBuilder.setSSLContext(sslContext);
        }

        RequestConfig requestConfig = RequestConfig.custom().setConnectTimeout(Math.toIntExact(options.getConnectionTimeout().toMillis())).setSocketTimeout(Math.toIntExact(options.getReadTimeout().toMillis())).setAuthenticationEnabled(true).build();
        httpClientBuilder.setDefaultRequestConfig(requestConfig);
        httpClientBuilder.setRedirectStrategy(new LaxRedirectStrategy());
        return httpClientBuilder;
    }

    // copied code from Spring vault core
    static SslConfiguration createSslConfiguration(VaultProperties.Ssl ssl) {
        SslConfiguration.KeyStoreConfiguration keyStore = SslConfiguration.KeyStoreConfiguration.unconfigured();
        SslConfiguration.KeyStoreConfiguration trustStore = SslConfiguration.KeyStoreConfiguration.unconfigured();
        if (ssl.getKeyStore() != null) {
            if (StringUtils.hasText(ssl.getKeyStorePassword())) {
                keyStore = SslConfiguration.KeyStoreConfiguration.of(ssl.getKeyStore(), ssl.getKeyStorePassword().toCharArray());
            } else {
                keyStore = SslConfiguration.KeyStoreConfiguration.of(ssl.getKeyStore());
            }

            if (StringUtils.hasText(ssl.getKeyStoreType())) {
                keyStore = keyStore.withStoreType(ssl.getKeyStoreType());
            }
        }

        if (ssl.getTrustStore() != null) {
            if (StringUtils.hasText(ssl.getTrustStorePassword())) {
                trustStore = SslConfiguration.KeyStoreConfiguration.of(ssl.getTrustStore(), ssl.getTrustStorePassword().toCharArray());
            } else {
                trustStore = SslConfiguration.KeyStoreConfiguration.of(ssl.getTrustStore());
            }

            if (StringUtils.hasText(ssl.getTrustStoreType())) {
                trustStore = trustStore.withStoreType(ssl.getTrustStoreType());
            }
        }

        return new SslConfiguration(keyStore, trustStore, ssl.getEnabledProtocols(), ssl.getEnabledCipherSuites());
    }

there are some missing parts, how inject VaultProperties data to configure the ClientOption and SslConfiguration before run the application as you its done in spring vault ? I will need to use multiple properties from spring configuration application as value of connectionTimeToLive, retry flag, .. and retrieve these properties does not seem be possible because spring context is not yes defined

Thank you.

[mp911de]

See also https://github.com/spring-projects/spring-vault/issues/760

[mp911de]

Applying customizations in the early application startup (before starting SpringApplication.run(…)) to not have access to PropertySources or Environment because these objects do not yet exist. Paging @mhalbritter from the Boot team, whether we have a chance to get hold of the config data in an early startup stage to apply customizations before ConfigDataLoaderis being activated.

[juriohacc]

So, do you mean that even for using the new method you added in https://github.com/spring-projects/spring-vault/issues/760 : createHttpAsyncClientBuilder, it's necessary to create ourself the ClientOptions and SslConfiguration, it cannot be provided by Spring Vault ?

Ok for the access of PropertySources/Environment , i'm going to see with @mhalbritter what it's possible to do.

[mhalbritter]

Normally an Environment is created for you when you call run. That means before run is called, there is no Environment available. However, you can use setEnvironment() on the SpringApplication to set a custom environment (this can be done before calling run). This environment is then used for startup, too. Not sure if this helps your usecase.

[mp911de]

Ideally, we could create an Environment with all the file-based and env-variable property sources that mimic Spring Boot's Environment for binding @ConfigurationProperties.

The alternative could be a customization hook in Spring Cloud Vault via InstanceSupplier that is leveraged by our ConfigDataLocationResolver or ConfigDataLoader to customize infrastructure components such as the HTTP client.

It basically boils down to being able to customize ConfigDataLoader before the application startup. I'm going to spend a few cycles on this approach to come up with a design idea.