SteelBridgeLabs / neo4j-gremlin-bolt

Apache License 2.0
0 stars 1 forks source link

outofmemory after put many entities and relations #61

Closed lhztop closed 6 years ago

lhztop commented 7 years ago

I find in Neo4jSession, save all vertices and edges in Hashmap: id->object. which will continiously increase the heap memory, eventually cause OOM, whatever the memory size.

rjbaucells commented 7 years ago

Yes, it is true that the session keeps a local section of the graph with the node/relations that have been loaded. Neo4jSession is meant to be used as a Unit Of Work, the duration of the session should be kept as small as possible, you cannot keep a Neo4JGraph instance alive for the duration of the application. This is the recommended use of the API:

// create graph instance
try (Graph graph = new Neo4JGraph(driver, vertexIdProvider, edgeIdProvider)) {
    // begin transaction
    try (Transaction transaction = graph.tx()) {
        // use Graph API to create, update and delete Vertices and Edges

        // commit transaction
        transaction.commit();
    }
}

The same behavior can be reproduced in other persistence frameworks; as an example, Hibernate session keeps a reference to all object instances that have been loaded. Not using the session as a short lived Unit of Work will cause the same problems.

It is possible that even using the Neo4JGraph instance as a Unit Of Work you get OOM errors by running statements like MATCH (n) RETURN n on large graphs. There is no enough memory to load the graph in memory.

What is your specific scenario?

Are you using the Neo4JGraph instance as a Unit of Work?

How many Nodes/Relations are you loading into a single Neo4JGraph instance?