vlingo / xoom-wire

The VLINGO XOOM platform SDK wire protocol messaging implementations, such as with full-duplex TCP and UDP multicast, and RSocket, using VLINGO XOOM ACTORS.
Mozilla Public License 2.0
13 stars 11 forks source link

Memory leak of ConsumerByteBufferPool objects #47

Closed pflueras closed 3 years ago

pflueras commented 3 years ago

This issue can be easily reproduced with vlingo-helloworld example.

vlingo-actors.properties configuration:

plugin.name.pooledCompletes = true
plugin.pooledCompletes.classname = io.vlingo.actors.plugin.completes.PooledCompletesPlugin
plugin.pooledCompletes.pool = 50
plugin.pooledCompletes.mailbox = queueMailbox

plugin.name.queueMailbox = true
plugin.queueMailbox.classname = io.vlingo.actors.plugin.mailbox.concurrentqueue.ConcurrentQueueMailboxPlugin
plugin.queueMailbox.defaultMailbox = true
plugin.queueMailbox.numberOfDispatchersFactor = 0
plugin.queueMailbox.numberOfDispatchers = 180
plugin.queueMailbox.dispatcherThrottlingCount = 1

# ... because of vlingo-xoom
plugin.name.slf4jLogger=true
plugin.slf4jLogger.classname=io.vlingo.actors.plugin.logging.slf4j.Slf4jLoggerPlugin
plugin.slf4jLogger.name=vlingo/xoom
plugin.slf4jLogger.defaultLogger=true

A test case with 150 concurrent threads has been used. Each thread performs following requests:

OutOfMemoryError shows up quite quickly. Heap size is 1024m (-Xmx1024m).

VaughnVernon commented 3 years ago

@pflueras As previously stated, this doesn't happen when using a maximum of 50 threads or the more common 12 - 16 threads. It might seem that the number of threads shouldn't matter, only the total simultaneous requests. Yet, I think the problem could be related to the amount of threads and memory to deal with the requests processed in a given timeframe. Given that the default the tests run with perhaps 12 to 16 total threads, using 200 is 12x - 16x more, it seems like instead of running with 1 GB heap that instead running with 12 GB - 16 GB heap would be fitting.

In other words, there may not be enough heap to handle both the simultaneous requests and the use of in-memory persistence.