BlueBrain / nexus

Blue Brain Nexus - A knowledge graph for data-driven science
https://bluebrainnexus.io/
Apache License 2.0
275 stars 73 forks source link

Investigate cassandra pub/subs in order to reduce polling intervals but retain performance #2462

Closed umbreak closed 3 years ago

umbreak commented 3 years ago

If this works, it should allow us to reduce the pressure on Cassandra (from multiple views streams parallel executions) while keeping the eventsByTag delay low.

imsdu commented 3 years ago

While pubsub can be used to lower latency as advertised in the akka-persistence-cassandra documentation, the tests show that enabling it and increasing significantly the polling interval is not a valid solution to reduce pressure on the primary store as it prevents indexing to work correctly.