Kafka Bridge not working as expected with 2 replicas

[shalin-hudda]

I want to have at least 2 replicas of Kafka Bridge to build in some redundancy in case of failure. Therefore, I am using the following yml with 2 replicas.

apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaBridge
metadata:
  name: my-bridge
  namespace: test
spec:
  authentication:
    type: tls
    certificateAndKey:
      secretName: my-user
      certificate: user.crt
      key: user.key
  tls:
    trustedCertificates:
      - secretName: my-cluster-cluster-ca-cert
        certificate: ca.crt
  bootstrapServers: 'my-cluster-kafka-bootstrap:9093'
  consumer:
    config:
      auto.offset.reset: earliest
      enable.auto.commit: true
  http:
    port: 8080
  producer:
    config:
      acks: 1
      delivery.timeout.ms: 300000
  replicas: 2

However, when the HTTP consumer tries to fetch records using the polling mechanism, the 1st replica responds as expected with the message but the 2nd replica responds with message not found and a 404 error response.

Following is my topic yaml

apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaTopic
metadata:
  name: topic-1
  namespace: test
  labels:
    strimzi.io/cluster: my-cluster
    strimzi.io/kind: Kafka
    strimzi.io/name: topic-1
spec:
  partitions: 1
  replicas: 1

Is this an expected response? Why is this happening?

[ppatierno]

Currently it's not possibile to have more than one replica of the Kafka bridge. This is mostly related on how the consumer part works because an instance of a corresponding Kafka consumer is created on the bridge when an HTTP client consumer connect. Each bridge has its own set of consumers; if you create a consumer on a bridge instance, that consumer has to connect to that instance not another one, there is no consumer instances shared across multiple bridge replicas. The only way to have multiple bridge is having multiple deployment of the bridge and maybe using a reverse proxy (i.e. nginx) in front of those instances in order to forward the traffic to the correct one with clients just connecting to the proxy address but specifying the bridge instance they want to hit.

[shalin-hudda]

Okay thank you for letting me know.

[RenGeng]

Hi @ppatierno, is it still the case in bridge 0.21.5 ?

[ppatierno]

Yes. It's not going to change.

[RenGeng]

So is there a way to disable the consumer part of the bridge ? Our use case is to use the bridge to act only as a producer.

[ppatierno]

It's not possible to disable one part or the other. I opened an issue upstream, maybe if you want you can contribute to it. It's not really and higher priority so contributions are welcome :-)