search

fcrepo-exts / fcrepo-camel-toolbox

A collection of ready-to-use messaging applications with fcrepo-camel
Apache License 2.0
13 stars 26 forks source link

JmsConsumer thread hogs CPU even when no updates are present in the queue #194

Open aciapetti opened 1 year ago

aciapetti commented 1 year ago

Using the Docker version of fcrepo-camel-toolbox (latest version) I got a 100% CPU usage for one of the container cores even when no update is present in the Fedora ActiveMQ queue.

My configuration of the toolbox is:

fcrepo.authHost=fcrepo fcrepo.authUsername=fedoraAdmin fcrepo.authPassword= ldpath.transform.path=file:///config/ldpath/teca.ldpath jms.brokerUrl=tcp://fcrepo:61616 solr.indexing.enabled=true solr.baseUrl=http://solr:8983/solr/fcrepo triplestore.indexing.enabled=true triplestore.baseUrl=http://fuseki:3030/fcrepo reindexing.rest.host=0.0.0.0

Output of the htop command: 1842434 root 20 0 13,5g 729244 25604 S 100,0 2,2 162:10.88 java -jar /usr/local/fcrepo-camel-toolbox/driver.jar -c /config/toolbox-configuration.properties (CPU is at 100% for at least one of my 8 cores)

Is it possible to lower the polling frequency on the JMS queue using the configuration? Is it in some way related to the Thread.onSpinWait() cycle in the Driver class?

Thanks Andrea

whikloj commented 5 months ago

Hi @aciapetti, it would appear it is related to the Thread.onSpinWait() loop as you mentioned.

--- Execution profile ---
Total samples       : 1306

--- 12840000000 ns (98.32%), 1284 samples
  [ 0] org.springframework.context.support.AbstractApplicationContext.isRunning
  [ 1] org.fcrepo.camel.toolbox.app.Driver.call
  [ 2] org.fcrepo.camel.toolbox.app.Driver.call
  [ 3] picocli.CommandLine.executeUserObject
  [ 4] picocli.CommandLine.access$1200
  [ 5] picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent
  [ 6] picocli.CommandLine$RunLast.handle
  [ 7] picocli.CommandLine$RunLast.handle
  [ 8] picocli.CommandLine$AbstractParseResultHandler.execute
  [ 9] picocli.CommandLine.execute
  [10] org.fcrepo.camel.toolbox.app.Driver.main

...

          ns  percent  samples  top
  ----------  -------  -------  ---
 12840000000   98.32%     1284  org.springframework.context.support.AbstractApplicationContext.isRunning
   120000000    0.92%       12  org.fcrepo.camel.toolbox.app.Driver.call
    30000000    0.23%        3  __psynch_cvwait
    20000000    0.15%        2  org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener
    10000000    0.08%        1  BiasedLocking::revoke_and_rebias
    10000000    0.08%        1  java.lang.AbstractStringBuilder.append
    10000000    0.08%        1  CollectedHeap::array_allocate
    10000000    0.08%        1  org.springframework.jms.listener.AbstractMessageListenerContainer.commitIfNecessary
    10000000    0.08%        1  java.util.concurrent.CopyOnWriteArrayList.iterator

If you have a recommendation, we will happily take it.

I'll add this to the list of things to have a look at. This app will need to change to be more spring-boot like with the latest Camel versions, but no one has been willing to take a look at that yet.