search

KonstantinCodes / messenger-kafka

Simple Kafka transport for Symfony Messenger.
MIT License
84 stars 35 forks source link

Split sender and receiver config to not include rebalance_cb for receiver. #70

Open klimser opened 1 year ago

klimser commented 1 year ago

$conf->setRebalanceCb() called either for consumer and for producer. It triggers warning from rdkafka CONFWARN:Configuration property rebalance_cb is a consumer property and will be ignored by this producer instance. Config divided into producer and consumer parts, ->setRebalanceCb() called for consumer config only.

andythorne commented 1 year ago

This seems like a sensible change - please can we get this merged @KonstantinCodes?

thePanz commented 1 year ago

Thanks for fixing this @klimser !

There are other configurations that are applied to the producer, but are only available for the consumer, see:

rdkafka#producer-1| [thrd:app]: Configuration property group.id is a consumer property and will be ignored by this producer instance
rdkafka#producer-1| [thrd:app]: Configuration property enable.auto.offset.store is a consumer property and will be ignored by this producer instance
rdkafka#producer-1| [thrd:app]: Configuration property rebalance_cb is a consumer property and will be ignored by this producer instance
klimser commented 1 year ago

Thanks for fixing this @klimser !

There are other configurations that are applied to the producer, but are only available for the consumer, see:

rdkafka#producer-1| [thrd:app]: Configuration property group.id is a consumer property and will be ignored by this producer instance
rdkafka#producer-1| [thrd:app]: Configuration property enable.auto.offset.store is a consumer property and will be ignored by this producer instance
rdkafka#producer-1| [thrd:app]: Configuration property rebalance_cb is a consumer property and will be ignored by this producer instance

You are right, but you can control all that properties via $options except rebalance_cb that is set inside the method (look at changes of this Pull request).

thePanz commented 1 year ago

Thanks for fixing this @klimser ! There are other configurations that are applied to the producer, but are only available for the consumer, see:

rdkafka#producer-1| [thrd:app]: Configuration property group.id is a consumer property and will be ignored by this producer instance
rdkafka#producer-1| [thrd:app]: Configuration property enable.auto.offset.store is a consumer property and will be ignored by this producer instance
rdkafka#producer-1| [thrd:app]: Configuration property rebalance_cb is a consumer property and will be ignored by this producer instance

You are right, but you can control all that properties via $options except rebalance_cb that is set inside the method (look at changes of this Pull request).

True, but if I set $options['group.id'] it will be applied to both the consumer and receiver :)