DotNet4Neo4j / Neo4jClient

.NET client binding for Neo4j
https://www.nuget.org/packages/Neo4jClient
Microsoft Public License
431 stars 146 forks source link

Basic client read queries lead to Java OOM on neo4j server #448

Closed dzindzinj closed 1 year ago

dzindzinj commented 2 years ago

Describe the bug We are running a basic neo4j database and we have found that when using neo4j client to query database, it fills up Java Heap on the database with ‘org.neo4j.bolt.transaction.StatementProcessorTxManager’ / 'statementProcessProviders` objects which often lead to Java OOM error on the server.

Following the heap dump tail we understand that it is a simple read query that runs periodically.

the query:

private readonly IGraphDbService _graphdbservice;

....

    {
        var result = await _graphdbservice.Query().Read.
            Match("(p:Paths)").
            Return(p => Return.As<int>("count(p)")).
            ResultsAsync.FirstOrDefault();
        return result > 0;
    }

client:

var client = new GraphClient(new Uri(url), username, password);

            try
            {
                client.ConnectAsync().Wait();
                _client = client;
            }
            catch (Exception ex)
            {
                throw new Exception($"Unable to connect to the graph database on {url} using the credentials supplied", ex);
            }

We've tried to replicate the problem locally using Apache Jmeter bolt connection and ran the query for millions of samples and simply could not reproduce the problem which leads us to believe it is Client driver that fails to clean up after itself.

Versions:

To Reproduce as above

Expected behaviour Could not find documentation of what these classes mean. We think these are left in memory in case we want to roll back a tx/commit statement, but these are basic read queries not write and should not be saved

Additional context Empty database has been upgraded from 2.3 all the way to the current latest without any problem. Unfortunately, we cannot use the new bolt protocol directly as we use Azure App service to host neo4j server.

Are we missing any additional methods in the query? How can we ensure that read transactions do not remain in server memory?

cskardon commented 1 year ago

I can't see how this is anything to do with the client.

I suggest you turn on the Query.log on your server and see what is actually using the memory. Also - your heap settings, pagecache - there are so many things that can cause that error - but it's not going to be an http connection.