search

SolaceProducts / solace-spring-cloud

An umbrella project containing all Solace projects for Spring Cloud
Apache License 2.0
22 stars 15 forks source link

Solace multiple consumers issue #196

Closed IagoSchlisting closed 1 year ago

IagoSchlisting commented 1 year ago

Hello,

During my tests, I have encountered a weird situation:

I've pre-filled my queue with lots of messages. Then, I started my spring cloud stream application to consume that queue, with .concurrency=50. Only a few threads/consumers processed the messages. Somehow, the messages that were in the queue are not being distributed among all consumers, but when publishing messages to this queue when my application is already started, all consumers process as expected.

My queue is not exclusive, not anonymous and neither uses polled consumers, also all consumers seem to be connected when my application starts. Any idea why this happens? Is this an expected behavior?

I'm looking forward to your return. Regards,

Mrc0113 commented 1 year ago

Hi @IagoSchlisting, Can you confirm that you are setting spring.cloud.stream.bindings.<bindingName>.consumer.concurrency ?

Also I'm going to move this to the solace-spring-cloud project since that is where the Cloud Stream binder resides.

Nephery commented 1 year ago

I've pre-filled my queue with lots of messages.

Try starting your application first, then filling your queue. Consumers are started sequentially, meaning that messages could be funneled to the consumers which start earlier if your test is using a pre-filled queue.

IagoSchlisting commented 1 year ago

Hi @IagoSchlisting, Can you confirm that you are setting spring.cloud.stream.bindings.<bindingName>.consumer.concurrency ?

@Mrc0113 Yes, I'm using scs concurrency configuration.

Also I'm going to move this to the solace-spring-cloud project since that is where the Cloud Stream binder resides.

Ok thank you!

IagoSchlisting commented 1 year ago

I've pre-filled my queue with lots of messages.

Try starting your application first, then filling your queue. Consumers are started sequentially, meaning that messages could be funneled to the consumers which start earlier if your test is using a pre-filled queue.

@Nephery filling the queue after starting application works good, so is this expected? I ask because for the same scenario using scs-rabbit it distributes fine among all consumers.

Nephery commented 1 year ago

Yes, this is expected.

If you had enough messages in your pre-filled queue, then once all consumers have started up, you should notice that message distribution will eventually reach a steady-state and use all the consumers.