Support declarative HTTP clients

[mhalbritter]

Spring Framework 6.0 introduces declarative HTTP clients. We should add some auto-configuration in Boot which supplies for example the HttpServiceProxyFactory.

We could even think about an annotation with which the HTTP interface can be annotated and then directly injected into a consumer (like @FeignClient).

[livk-cloud]

I think this one can try to inject IOC by myself For example I do

public class HttpServiceFactory implements BeanFactoryAware, ImportBeanDefinitionRegistrar, ResourceLoaderAware {

    private final HttpServiceProxyFactory proxyFactory;

    private BeanFactory beanFactory;

    private ResourceLoader resourceLoader;

    public HttpServiceFactory() {
        WebClient client = WebClient.builder().build();
        this.proxyFactory = HttpServiceProxyFactory.builder(new WebClientAdapter(client)).build();
    }

    @Override
    public void registerBeanDefinitions(@NonNull AnnotationMetadata importingClassMetadata,
            @NonNull BeanDefinitionRegistry registry) {
        List<String> packages = AutoConfigurationPackages.get(this.beanFactory);
        Set<Class<?>> typesAnnotatedClass = findByAnnotationType(HttpExchange.class, resourceLoader,
                packages.toArray(String[]::new));
        for (Class<?> exchangeClass : typesAnnotatedClass) {
            BeanName name = AnnotationUtils.getAnnotation(exchangeClass, BeanName.class);
            String beanName = name != null ? name.value()
                    : CaseFormat.UPPER_CAMEL.to(CaseFormat.LOWER_CAMEL, exchangeClass.getSimpleName());
            registry.registerBeanDefinition(beanName, getBeanDefinition(exchangeClass));
        }
    }

    private <T> BeanDefinition getBeanDefinition(Class<T> exchangeClass) {
        return new RootBeanDefinition(exchangeClass, () -> proxyFactory.createClient(exchangeClass));
    }

    @Override
    public void setResourceLoader(@NonNull ResourceLoader resourceLoader) {
        this.resourceLoader = resourceLoader;
    }

    @Override
    public void setBeanFactory(@NonNull BeanFactory beanFactory) throws BeansException {
        this.beanFactory = beanFactory;
    }

public Set<Class<?>> findByAnnotationType(Class<? extends Annotation> annotationClass,
            ResourceLoader resourceLoader, String... packages) {
        Assert.notNull(annotationClass, "annotation not null");
        Set<Class<?>> classSet = new HashSet<>();
        if (packages == null || packages.length == 0) {
            return classSet;
        }
        ResourcePatternResolver resolver = ResourcePatternUtils.getResourcePatternResolver(resourceLoader);
        CachingMetadataReaderFactory metadataReaderFactory = new CachingMetadataReaderFactory(resourceLoader);
        try {
            for (String packageStr : packages) {
                packageStr = packageStr.replace(".", "/");
                Resource[] resources = resolver.getResources("classpath*:" + packageStr + "/**/*.class");
                for (Resource resource : resources) {
                    MetadataReader metadataReader = metadataReaderFactory.getMetadataReader(resource);
                    String className = metadataReader.getClassMetadata().getClassName();
                    Class<?> clazz = Class.forName(className);
                    if (AnnotationUtils.findAnnotation(clazz, annotationClass) != null) {
                        classSet.add(clazz);
                    }
                }
            }
        }
        catch (IOException | ClassNotFoundException e) {
            throw new RuntimeException(e);
        }
        return classSet;
    }

}

/**
 * Just used to set the BeanName
 */
@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
public @interface BeanName {

    String value();

}

What is your opinion on this

[spencergibb]

What would an additional annotation bring beyond @HttpMapping?

[rstoyanchev]

The underlying client may need to be configured with different base URL, codecs, etc. That means detecting @HttpExchange annotated interfaces and declaring beans for them could become inflexible if it's based on the same underlying client setup.

Given an HttpServiceProxyFactory, it's trivial to create a proxy, so don't see a lot of gain from automating that. What I am thinking about though is that it should be possible for Boot to create and auto-configure one HttpServiceProxyFactory instance that can then be combined that with different client setups.

Currently HttpServiceProxyFactory takes HttpClientAdapter in the constructor, but we could make a change to allow it to be passed to an overloaded createClient(Class<?>, HttpClientAdapter) method. So you could inject the same HttpServiceProxyFactory anywhere, and either use it with the default client setup (based on WebClientCustomizer and WebClient.Builder), or optionally, also inject WebClient.Builder and use a client setup that deviates from the default.

[mhalbritter]

I played around with it here.

The auto-configuration supplies a bean of type HttpServiceProxyFactory, which a user can then use to create the proxy from the interface. The base url for the client is settable via @HttpExchange and is not configured on the factory.

[rstoyanchev]

Thanks, this helps me to move my thought process forward. I see now it is necessary to separate more formally the HttpServiceProxyFactory from the underlying client.

I've an experiment locally where HttpServiceProxyFactory expects the HttpClientAdapter to be passed in every time createClient is called. Separately, there is a WebClientServiceProxyFactory that is created with an HttpServiceProxyFactory and a WebClient and exposes a createClient with just the proxy interface.

The Boot auto-config could then declare a single HttpServiceProxyFactory bean, and applications would create any number of WebClientServiceProxyFactory beans, each delegating to the same HttpServiceProxyFactory and the WebClient configured for a specific remote.

[rstoyanchev]

After a few different experiments, I think trying to have one HttpServiceProxyFactory for many client instances brings additional complexity with little gain. The easiest to understand model remains, one HttpServiceProxyFactory for one client. It's reasonably simple even without any help from Boot:

@Bean
HttpServiceProxyFactory httpServiceProxyFactory1(WebClient.Builder clientBuilder) {
    WebClient client = clientBuilder.baseUrl("http://host1.com").build();
    return new HttpServiceProxyFactory(new WebClientAdapter(client));
}

@Bean
HttpServiceProxyFactory httpServiceProxyFactory2(WebClient.Builder clientBuilder) {
    WebClient client = clientBuilder.baseUrl("http://host2.com").build();
    return new HttpServiceProxyFactory(new WebClientAdapter(client));
}

A couple of extra shortcuts on WebClientAdapter could make this a one-liner:

@Bean
HttpServiceProxyFactory httpServiceProxyFactory1(WebClient.Builder clientBuilder) {
    return WebClientAdapter.createProxyFactory(clientBuilder.baseUrl("http://host1.com"));
}

@Bean
HttpServiceProxyFactory httpServiceProxyFactory2(WebClient.Builder clientBuilder) {
    return WebClientAdapter.createProxyFactory(clientBuilder.baseUrl("http://host2.com"));
}

If we settle on the above as the expected configuration, then I think it's not essential to have any Boot auto-config to start, although some ideas may still come along. Perhaps, specifying baseUrl's in properties, which would allow Boot to create the above beans?

[wilkinsona]

Thanks, Rossen.

Perhaps, specifying baseUrl's in properties, which would allow Boot to create the above beans?

We don't yet support specifying multiple property values to auto-configure multiple beans anywhere in Boot. It's something that we'd like to do, but it's a complex and wide-ranging topic. https://github.com/spring-projects/spring-boot/issues/15732 is tracking auto-configured multiple DataSources, for example.

Having discussed this today, we don't think there's anything to do in Boot at this time. We can revisit this if the picture changes for auto-configuring multiple beans.

[rstoyanchev]

Note that there is now https://github.com/spring-projects/spring-framework/issues/29296, which will likely give us a better model for dealing with multiple HttpServiceProxyFactory instances for different remotes.

[bclozel]

Reopening because of changes made in spring-projects/spring-framework#29296 Spring Boot could contribute a pre-configured HttpServiceProxyFactory.Builder to the context so developer can build their own client from it.

[hannah23280]

With regard to this nice tutorial on HTTP Interface https://softice.dev/posts/introduction_to_spring_framework_6_http_interfaces/,

I don't quite understand. Why do developer need to manually write a @Bean method that will return the proxy bean (which implement the interface) especially if we are using spring boot? I recall using @FeignClient, I do not have to define any proxy bean for it, so I presume spring boot will do for us.

Also why would one use Http Interface over @FeignClient?

[DanielLiu1123]

We could even think about an annotation with which the HTTP interface can be annotated and then directly injected into a consumer (like @FeignClient).

I think we need an annotation like @EnableFeignClients rather than @FeignClient.

We can already know whether an interface is a http client through @HttpExchange, we need an annotation to scan the interfaces and register beans (like @EnableFeignClients).

Here‘s my workaround.

[alsikorski]

I think people have gotten used to using the Feign client approach.

Here you can find very similar aproach: Exchange client

It is really simple to use.

[maciejwalkowiak]

I have prototyped following approach, that reduces the boilerplate to minimum:

@HttpClient annotation to mark interface as an http client and add option to set the WebClient bean name to use.

@HttpClient("todo-client")
public interface TodoClient {
    @GetExchange("/todos")
    List<Todo> get();
}

This annotation is processed by an registrar implementing ImportBeanDefinitionRegistrar that registers bean definition for each http client with HttpServiceProxyFactory and WebClientAdapter creating an adapter for a WebClient with a name from the annotation.

Creating WebClient instances from the environment

Considering that many web clients are relatively simple, there is a common set of properties that can be set with simple properties: url, basic auth, timeouts etc.

Given this, there is an option to create WebClients through yaml/properties like this:

http.clients:
    todo-client:
        url: https://jsonplaceholder.typicode.com
    bar:
        url: http://foo/bar

If you believe it makes sense I can prepare a PR or if it's too early to say I can release it as a separate project that will get deprecated once Spring Boot has similar functionality built in.

Update:

The library is available on Maven Central: https://github.com/maciejwalkowiak/spring-boot-http-clients

[OlgaMaciaszek]

Another proposed implementation: https://github.com/joshlong/declarative-client-registration by @joshlong .

[DanielLiu1123]

Let me introduce once again to httpexchange-spring-boot-starter, which is probably the most comprehensive implementation I could find.

This project is entirely configuration-driven and can achieve the same functionalities as Spring Cloud OpenFeign without introducing any external annotations. Including setting different baseUrl/timeout for each client, integration with Spring Cloud LoadBalancer, dynamic refresh, and more.

http-exchange:
  base-packages: [com.example]
  connect-timeout: 1000
  read-timeout: 3000
  client-type: rest_client
  channels:
    - base-url: http://order
      clients:
        - com.example.order.api.*
    - base-url: http://user
      read-timeout: 5000
      clients:
        - com.example.user.api.*

The main goals of this project:

• Promote the use of @HttpExchange as a neutral annotation to define API interfaces.
• Provide a Spring Cloud OpenFeign like experience for Spring 6.x declarative HTTP clients.
• Support @RequestMapping based annotations (easy to migrate from Spring Cloud OpenFeign).
• Not introduce external annotations, easy to migrate to other implementations.

It's definitely worth a try!

[hannah23280]

Let me introduce once again to httpexchange-spring-boot-starter, which is probably the most comprehensive implementation I could find.

This project is entirely configuration-driven and can achieve the same functionalities as Spring Cloud OpenFeign without introducing any external annotations. Including setting different baseUrl/timeout for each client, integration with Spring Cloud LoadBalancer, dynamic refresh, and more.

http-exchange:
  base-packages: [com.example]
  connect-timeout: 1000
  read-timeout: 3000
  client-type: rest_client
  channels:
    - base-url: http://order
      clients:
        - com.example.order.api.*
    - base-url: http://user
      read-timeout: 5000
      clients:
        - com.example.user.api.*

The main goals of this project:

• Promote the use of @HttpExchange as a neutral annotation to define API interfaces.
• Provide a Spring Cloud OpenFeign like experience for Spring 6.x declarative HTTP clients.
• Support @RequestMapping based annotations (easy to migrate from Spring Cloud OpenFeign).
• Not introduce external annotations, easy to migrate to other implementations.

It's definitely worth a try!

Great Effort!! Wish the spring boot will include your starter as an official one!

[hannah23280]

So is there any plan or any work in progress to include this feature natively?

[OlgaMaciaszek]

We're discussing it within the team now. We will update here once decision's been taken.

[FilipBehnke]

@OlgaMaciaszek are there any updates on this topic?

[OlgaMaciaszek]

Internal POC planned for this month to discussed with the team. Will post any updates here.

[XhstormR]

@OlgaMaciaszek Since spring-cloud-openfeign is going to enter maintenance mode, will this become the alternative options to openfeign, any update for now?

[ZaheerUdDeen]

Hey Folks, any updates here. when the declrative approach will be available natively?

[OlgaMaciaszek]

Working on POC at this point. Once that's done we'll able to hold a discussion on adding it to a backlog of a specific release. Will post any relevant updates here.

[yuexueyang]

@OlgaMaciaszek Thank you for introduce me to this post. httpexchange-spring-boot-starter is really helpful, but may be it still have a small difference from what I want. That's we always configure the base url in database, not in property files, and we need to obtain the base url during every call to the HttpExchange interface to ensure that we use the proper value for each customer (Yes, different customer may use different base url, and the remote site is not constucted by ourself)

[xtyuns]

不同的客户可能使用不同的基 URL

Maybe you can do it with an org.springframework.http.client.ClientHttpRequestInterceptor

[spencergibb]

Always get your configuration from the spring environment and use different configuration sources to load from a property or DB or whatever

[yuexueyang]

不同的客户可能使用不同的基 URL

Maybe you can do it with an org.springframework.http.client.ClientHttpRequestInterceptor In my experience, this interceptor can only modify header values, not url.

[yuexueyang]

Always get your configuration from the spring environment and use different configuration sources to load from a property or DB or whatever

Thank you for your advice, what you mean is to configurer Proxy instance per customer for specified HttpExchange at startup, like Bean instance A for customer A, Bean instance B for customer B, and so on. Is this understanding correct?

[xtyuns]

In my experience, this interceptor can only modify header values, not url.

You can replace the request, just like the example with kotlin:

val urlPrefixedInterceptor = ClientHttpRequestInterceptor { request, body, execution ->
    execution.execute(object : HttpRequest by request {
        override fun getURI(): URI {
            return URI.create("https://example.org${request.uri}")
        }
    }, body)
}