compaction is CPU cannot be used efficiently

[kamijin-fanta]

I was investigating the problem of compaction very slow when I created lucene index.

# nodetool compactionstats -H

id                                   compaction type       keyspace   table   completed total    unit  progress
c0560fd0-a43f-11e7-bb0a-99ea385ac574 Secondary index build ex         message 1.32 MiB  1.76 MiB bytes 74.80%
Active compaction remaining time :        n/a

after 30 second

# nodetool compactionstats -H

id                                   compaction type       keyspace   table   completed total    unit  progress
c0560fd0-a43f-11e7-bb0a-99ea385ac574 Secondary index build ex         message 1.37 MiB  1.76 MiB bytes 77.60%
Active compaction remaining time :        n/a

Since indexing_threads is unspecified, I guess that the same number of CPU cores as 6 is specified. However, the usage rate of the CPU never exceeds 100% / 600%.

I thought it suspicious and looked at the situation of the thread

use tools: https://visualvm.github.io/

Only one thread is running at the same time...

Does Lucene Incexer have conditions to use concurrent threads?

---

memo

enable rmi server

echo 'JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=192.168.10.181"' >> /etc/cassandra-env.sh

spec

• cassandra
  • version: 3.11.0
  • VirtualNode: Enabled
  • Nodes: 5
• cassandra-lucene-index version: 3.11.0
• OS: Ubuntu 14.04.4 LTS
• JVM: OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-1~bpo8+1-b11)
• CPU: Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz (6 core use)
• RAM: 12GB

[kamijin-fanta]

When multiple compactions were operated at the same time, it was slightly improved. However, it can not be said that 100% of resources can be used.

It seems to be meaningless to specify indexing_threads larger thanconcurrent_compactors. In Cassandra's configuration, concurrent_compactors are generally 2. Are you specifying big numbers?

https://docs.datastax.com/en/cassandra/3.0/cassandra/configuration/configCassandra_yaml.html#configCassandra_yaml__concurrent_compactors

[kamijin-fanta]

I was misunderstanding. I guessed the indexer is a bottleneck. But it was a CompactionExecutor.

It is my own answer.

• Compactions referring to the same Lucene partition interfere.
• Performance improves if specify a larger number of partitions per node.
  • ex: 'partitioner': '{type: "token", partitions: 64}'
• indexing_threads is sufficient by default.

I hope this will be helpful to developers. Finally, thanks for the wonderful plugin!