RdKafkaConsumer subscribes to queue name instead of topic name

[floplus]

I found something i THINK is wrong. As I am pretty new to enqueue and kafka i don't know if it is really a bug, but it bugges/d me. :)

I tried to connect two symfony applications via kafka and didn't get it working until i followed how the consumer is build.

In QueueInteropTransport.php->getConsumer() the $queue is build from the $destination which has topic and queue set either from default options or from Envelope.

In RdKafkaContext the queue is build by passing the queue name to RdKafkaTopic. Then in createConsumer a new RdKafkaConsumer is created by passing the Destination (which is the Queue wihich is the RdKafkaTopic) as third parameter (which is a RdKafkaTopic).

The RdKafkaConsumer then subscribes (line 90) to the Queue name.

Shouldn't it be the topic name?

Now essentially queue and topic name MUST be identical to make the consumer consume messages from a queue and topic.

[Steveb-p]

True, but you can't really create consumers on the fly when running messenger:consume. Subscribed topic has to come from configuration. The ability to set a target Kafka topic is, in practice, only to allow passing message to a different topic / bus / application.

Theoretically regex can be used to subscribe to multiple kafka topics at once, but it's not really used frequently.

[floplus]

@Steveb-p i think i didn't make my concern clear. I know that with messenger:consume only the default values from configuration can be taken, but the topic the consumer subscribes to is the queue name not the topic name. But you can configure topic and queue separately (which you effectly can't because of the current implementation).