spring-cloud / spring-cloud-stream

Framework for building Event-Driven Microservices
http://cloud.spring.io/spring-cloud-stream
Apache License 2.0
969 stars 595 forks source link

Kafka Reactive Binder does not work with multiple bindings #2965

Open akoufa opened 1 week ago

akoufa commented 1 week ago

When using the Kafka Reactive Binder to consume messages from two different Kafka Cluster the app start fails with onfigReactiveWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context

I have a similar setup like the following:

spring:
  cloud:
    stream:
      binders:
        kafka1:
          type: kafka
          environment:
            spring:
              cloud:
                stream:
                  kafka:
                    binder:
                      brokers: kafka-cluster-1:9092
                      configuration:
                        security.protocol: SSL
                        ssl.truststore.location: /path/to/truststore.jks
                        ssl.truststore.password: yourpassword
        kafka2:
          type: kafka
          environment:
            spring:
              cloud:
                stream:
                  kafka:
                    binder:
                      brokers: kafka-cluster-2:9092
                      configuration:
                        security.protocol: SSL
                        ssl.truststore.location: /path/to/truststore.jks
                        ssl.truststore.password: yourpassword

      bindings:
        input1-in-0:
          destination: topic1
          binder: kafka1
        input2-in-0:
          destination: topic2
          binder: kafka2

and

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-stream</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-stream-binder-kafka-reactive</artifactId>
</dependency>

I tried to add the dependency:

  <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-stream-binder-kafka</artifactId>
    </dependency>

so that the configuration works but then the reactive part is ignored and it seems that the messages are consumed on a non reactive thread container-0-C-1 instead of nc-read-local-1 .

sobychacko commented 1 week ago

Reactive Kafka binder does not have support for multi binders (or at least we haven't tested this yet). We can add this as an enhancement item for the next release. If you are up to it, we welcome any PR contributions.