Closed jindrazak closed 5 years ago
Hello @jindrazak and thank you for reporting this issue.
I agree that Neo4j crashing like this is undesirable behaviour, and would probably label this as a bug. However, the usage pattern that you're exemplifying is not one that we're optimising for. The library is mainly intended to be used for large graphs where executing the algorithm is a substantial effort, and something typically is done only a very small number of times.
If I'm reading your setup correctly, it seems as though the algorithms was successfully executed at least 1000 times before crashing, which I would label as good enough. We don't expect users to run the algorithm that many times in lockstep. If it would crash at 10 or 50 subsequent runs, then that would be worse, but after 1000 runs I think this doesn't reach highest priority.
Are you seeing this behaviour for much fewer runs if your graph is larger?
Ok, so I generated more nodes and relationships to my graph using this simple algorithm:
int nodes = 10;
//create nodes
for (int i = 0; i < nodes; i++) {
String createQuery = "CREATE (x:MyNode{id:" + i + "})";
session.run(createQuery);
}
//create relationships
for (int i = 0; i < nodes - 1; i++) {
String createQuery = "MATCH (a {id:" + i + "}), (b {id:" + (i + 1) + "}) CREATE (a)-[:rel]->(b)";
session.run(createQuery);
}
It creates a graph that looks like this: (a)--(b)--(c)-- ... --(y)--(z)
Then, I made a couple of measurements about how many iterations it can process before it crashes. Here is the data:
Number of nodes | Number of PR iterations before crash |
---|---|
1 | 1040 |
2 | 521 |
3 | 351 |
4 | 266 |
5 | 213 |
6 | 175 |
7 | 150 |
8 | 132 |
9 | 117 |
10 | 105 |
@jindrazak Where do you see that Neo4j crashes? The log shows that your connections are being terminated, but that is likely because of how you use the driver. You run a bunch of queries but you never consume the results. At the very least you should call StatementResult#consume()
. See the results section in the driver manual. Also, you should use transaction functions instead of auto-commit transactions, to get automatic retries for example.
What are you trying to accomplish anyway? I don't see what the use case is for running pagerank hundreds or thousands of times for 1-10 node graphs.
Using Java driver I want to run Pagerank repeatedly in a loop. However, after certain number of iterations neo4j crashes.
Steps to reproduce:
Here is my Java class:
Java log:
Neo4j logs:
debug.log: https://pastebin.com/5EeZwKep neo4j.log: https://pastebin.com/DwAYnw6P