OOM while removing all incorrect SSPublication records.

[olegs]

Removing all the SSPublications which were uploaded from incorrect tsk file by command

MATCH (n:SSPublication) DELETE n;

leads to the OOM exception.

Additional memory configuration is required as described here: https://neo4j.com/docs/operations-manual/current/performance/memory-configuration

Complete exception log:

2019-09-20 08:53:28.463+0000 INFO  ======== Neo4j 3.5.9 ========
2019-09-20 08:53:28.491+0000 INFO  Starting...
2019-09-20 08:53:42.268+0000 INFO  Bolt enabled on 0.0.0.0:7687.
2019-09-20 08:53:45.367+0000 INFO  Started.
2019-09-20 08:53:47.064+0000 INFO  Remote interface available at http://localhost:7474/
2019-09-20 08:58:46.847+0000 WARN  The client is unauthorized due to authentication failure.
2019-09-20 08:59:01.870+0000 WARN  The client is unauthorized due to authentication failure.
2019-09-20 08:59:01.896+0000 WARN  The client is unauthorized due to authentication failure.
2019-09-20 08:59:52.480+0000 WARN  The client is unauthorized due to authentication failure.
2019-09-20 11:06:39.727+0000 WARN  The client is unauthorized due to authentication failure.
2019-09-20 20:25:53.287+0000 WARN  The client is unauthorized due to authentication failure.
2019-09-21 07:13:50.204+0000 WARN  The client is unauthorized due to authentication failure.
2019-09-24 07:53:01.401+0000 WARN  The client is unauthorized due to authentication failure.
Exception in thread "Neo4j UDC Timer" Exception in thread "qtp786470345-77" Exception in thread "neo4j.Scheduler-1"
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "neo4j.Scheduler-1"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "neo4j.VmPauseMonitor-1"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp786470345-77"
Exception in thread "neo4j.VmPauseMonitor-1" java.lang.OutOfMemoryError: Java heap space
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Neo4j UDC Timer"

Exception in thread "qtp786470345-91" java.lang.OutOfMemoryError: Java heap space
Exception in thread "qtp786470345-293" Exception in thread "qtp786470345-128" Exception in thread "qtp786470345-92" java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
2019-09-24 21:08:38.466+0000 WARN  Unexpected thread death: org.eclipse.jetty.util.thread.QueuedThreadPool$2@36472783 in QueuedThreadPool[qtp786470345]@2ee095c9{STARTED,18<=17<=38,i=3,q=0}[ReservedThreadExecutor@7d571516{s=0/3,p=0}] Java heap space
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp786470345-89"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Session-HouseKeeper-7ebaf0d"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp786470345-296"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "neo4j.BoltWorker-3 [bolt]"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp786470345-297"
Exception in thread "qtp786470345-295" java.lang.OutOfMemoryError: Java heap space
Exception in thread "neo4j.BoltWorker-4 [bolt]" java.lang.OutOfMemoryError: Java heap space
Exception in thread "neo4j.BoltNetworkIO-57" 2019-09-25 06:12:30.066+0000 WARN  Unexpected thread death: org.eclipse.jetty.util.thread.QueuedThreadPool$2@36472783 in QueuedThreadPool[qtp786470345]@2ee095c9{STARTED,18<=13<=38,i=1,q=0}[ReservedThreadExecutor@7d571516{s=0/3,p=0}] Java heap space
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp786470345-300"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "neo4j.BoltNetworkIO-11"

[olegs]

This can be fixed by

• adding --volume=$HOME/neo4j/config:/var/lib/neo4j/conf to the command line launcher
• modification of $HOME/neo4j/config/neo4j.conf. Additional config:

  dbms.memory.heap.initial_size=64G
  dbms.memory.heap.max_size=64G

[ctrltz]

This does not seem to be the case, but in general one would rather use DETACH DELETE to delete nodes with their relationships - see https://neo4j.com/docs/cypher-manual/current/clauses/delete/

OOM error is also relevant for crawlers, another possible solution is APOC library with periodic commit extension: https://neo4j.com/docs/labs/apoc/current/graph-updates/periodic-execution/#commit-batching