Enabling lazy initialization breaks Kafka Listeners

[PatrikScully]

I've been trying to optimize startup times by enabling spring.main.lazy-initialization. However, after making this change, I've encountered an issue with Kafka listeners. Specifically, the listeners stop receiving messages.

I attempted to explicitly set the @Lazy(false) annotation on the Kafka listener components, this hasn't resolved the problem. While it appears that the listener beans are created at startup (as expected with @Lazy(false)), they still do not receive any messages from Kafka.

I use Spring Boot 3.2.2 with Spring Cloud Stream 4.1.0 and I configure my Kafka listeners from the property yml like this:

spring:
  cloud:
    config:
      allowOverride: true
      overrideNone: true
      overrideSystemProperties: false
    function:
      definition: >-
        listenExample;
    stream:
      bindings:
        listenExample-in-0:
          destination: example-topic
          group: example-group
          consumer:
            use-native-decoding: true
      kafka:
        bindings:
          listenExample-in-0:
            consumer:
              enableDlq: true
              dlqName: example-error-topic
              dlqPartitions: 1

The component class:

@Component("listenExample")
@Lazy(false)
@RequiredArgsConstructor
@Transactional
public class ExampleListener implements Consumer<Message<String>> {

    @Override
    public void accept(Message<String> message) {
        // do something
    }
}

The expected behavior could be the possibility of loading these stream components in a lazy initialization environment using a config property, for example.

[sobychacko]

Could you put this in a small sample application so we can reproduce the issue?

[PatrikScully]

Could you put this in a small sample application so we can reproduce the issue?

@sobychacko Yes, of course, you can find here: https://github.com/PatrikScully/spring-cloud-kafka-lazy-init-sample

[sobychacko]

The problem is that when you set spring.main.lazy-initialization to true, it does not eagerly instantiate several beans that Spring Cloud Stream instantiates otherwise for framework-related tasks such as establishing the bindings. To activate those beans, you must ensure that the beans in org.springframework.cloud.stream.function.FunctionConfiguration are started with lazy init false. We always instantiate those beans eagerly; however, when you set the lazy-initialization property to true, that affects these beans too. We can consider adding @Lazy(false) on FunctionConfiguration. However, we need to do some more due diligence on this to make sure that there are no side-effects from doing this. As a workaround, you can try this technique in your application:

@Bean
static BeanFactoryPostProcessor ensureEagerManagementServerInitializationPostProcessor() {
        return (beanFactory) -> {
            for (String definitionName : beanFactory.getBeanDefinitionNames()) {
                if (definitionName.startsWith("functionBindingRegistrar") || definitionName.startsWith("functionInitializer")) {
                    beanFactory.getBeanDefinition(definitionName).setLazyInit(false);
                }
            }
        };
}

See this somewhat related Boot issue comment: https://github.com/spring-projects/spring-boot/issues/16184#issuecomment-471219121

If your application has suppliers (which you don't have, based on the sample application you shared, you also need to set lazy-init to false on the supplierInitializer bean.

As an aside, if your intention is to optimize startup time, have you looked into Spring Cloud Stream's AOT support for creating GraalVM native applications?

[PatrikScully]

This workaround seems to work correctly. Thank you! I think it would be better to put the @Lazy(false) annotation on the config, because these listeners should start anyway at startup.

I added the @Lazy(false) annotation to the autoconfig, and I tried with my sample application, and it's working fine as expected.

As an aside, if your intention is to optimize startup time, have you looked into Spring Cloud Stream's AOT support for creating GraalVM native applications?

Yes, it would be my main goal, but it needs a bigger refactoring to my existing project, so first I'm trying to make little performance tunings at startup.