Best Practices for finding the balance between too many and too few consumers?

[EdLichtman]

Hi Havret,

I appreciate the Apache Artemis Net Client Library you've provided.

When I have a Singleton Producer and Singleton Consumer, after about 200 messages, I stop receiving messages on the consumer side and Artemis has the following error:

AMQ212037 Connection failure to /{ip_address} has been detected: An existing connection was forcibly closed by the remote host [code=GENERIC_EXCEPTION]

However, when I create and dispose a new Consumer every message, I get past the 200 messages, only to find that I've broken Java GC with an out of memory exception.

Finally, when I create and dispose a new Producer every publish, I get like 10 messages in from the Producers before Artemis comes crashing down with the out of memory exception.

Assuming that you can't really wait any time before polling the message Queue, or before pushing a message to the queue, do you have any suggestions for how to make this work?

[Havret]

Hi @EdLichtman,

Based on your description, it seems that you're not acknowledging the messages, which leads to the blocking of your consumers. You can read about it more here --> https://havret.github.io/dotnet-activemq-artemis-client/docs/consumer-credit. Unacknowledged messages are never removed from the broker memory and may lead to an OOM exception. ActiveMQ Artemis was not designed to handle large message depths.

As a general rule of thumb, you should not create producers/consumers to handle a single message. This is a common anti-pattern that leads to performance degradation almost every single time when applied.

I hope that helps, Havret

[EdLichtman]

That's it! I changed it to 500 and the number went above 200!

I then ran consumer.AcceptAsync().ConfigureAwait() and I was able to watch it get up to 15,000 no problem. thank you so much, and for the explanatino as to why.