Possible memory leak

[lesobre]

Hello, When i used ravendbclient 4.2.106 and previous i have a lot of jsondocumentbuilder

BasicDirectFacade.txt DocumentStoreHolder.txt

Please help me !

[ayende]

That isn't actually a memory leak, mind. These are pooled instances that we avoid allocating many times.

[lesobre]

Thanks , i see i havent this problem with simple object but with complex the memory reach at 1 gb what determined that an object is pooled, or it is possible to clear the pool ? thanks

[ayende]

The number of objects that you push through each session, mostly. Also the number of sessions and document stores that you have.

We actually optimized this part of the code fairly heavily to reduce further memory consumption. You can check the 5.0 nightlies to see if that helps for you

[lucascebertin]

I see the same here, large objects returned from Queries, I got a lot of memory being used... Don't know how to deal with it. Using version 4.2, 5.0 and the latest from myget, same results :cry:

[ayende]

See here, might be relevant: https://github.com/ravendb/ravendb/pull/11265

Do you have a way for us to reproduce this?

[lucascebertin]

I'll try to create a repo and share with you

[lucascebertin]

Ok, just pushed here: sample

It's a simple console app, insert 1000 of registries then loop query for all. The memory starts to grow, from 30MB to 1.3GB ~ .14GB, then drops to 900MB ~ 1GB, then finishes.

Maybe irrelevant but I'm on Linux Manjaro, 20.1.1

[lucascebertin]

May i help with something else? Any news, please, let me know and thanks for your help

[ppekrol]

Hi @lucascebertin

I will be looking at the issue today in the evening or tomorrow afternoon. I will keep you posted. We made some enhancements that are available in 5.0-nightly Client API and I'm very curious how they affect your case.

[lucascebertin]

@ppekrol, thanks! Are they available at myget?

I just tried it when @ayende told me about, it's pinned 5.1.0-nightly-20201015-0935 This line in csproj if helps

Let me know if I could help with anything.

[ppekrol]

@lucascebertin Yes, they are available here: https://myget.org/feed/ravendb/package/nuget/RavenDB.Client/5.0.4-nightly-20201019-0907

[lucascebertin]

I'll try it and share the results, thanks for the tip

[lucascebertin]

@ppekrol, my scenario changed a lot after using this nightly and @ayende, sorry, I misunterstood your advice previously. Thought that using nightly 5.1.0 was the improvements.

Sharing the results: My app memory starts to grow, but sometimes shrinking, after the dispose automatically I got 130MB used, no more 1GB+. Should I try it or wait for a release?

[ppekrol]

@lucascebertin The new behavior that you described is something that I would actually expected to see. And I'm really happy that this is the case. I've checked all the changes that we made in 5.0 between last stable and today. The fixes there are related to memory management and some corner cases. If it is OK with you, I would recommend sticking with the nightly till our release around 9th of November.

[lucascebertin]

Perfect, @ppekrol, going to stick with it. Probably we'll publish this upgrade because we have a few PODs dying. Thanks!!

[ppekrol]

Great.

I will close down this issue. Please reopen if needed.

[lucascebertin]

@ppekrol, in an asp.net core app, I'm seeing the memory slowly grow up but no shrinking.

This could be a problem at my side, so, could you guys check my DI setup, please?

        public static IServiceCollection AddRavenDB(this IServiceCollection builder, RavenDBOptions options)
        {
            var store = new DocumentStore
            {
                Urls = options.Urls,
                Database = options.Database,
                Certificate = options.Certificate,
            };

            store.Initialize();
            builder.AddSingleton<IDocumentStore>(store);

            builder.AddScoped<IAsyncDocumentSession>(serviceProvider =>
                serviceProvider
                    .GetService<IDocumentStore>()
                    .OpenAsyncSession()
            );

            return builder;
        }

Is it ok or should I change something here?

[ppekrol]

Hi @lucascebertin

I do not see anything suspicious here. What you need to be aware of is that RavenDB Client API contains caches (e.g. for HTTP) and memory pool (for memory management) and during the execution of the application the memory can be retained there for future use.

The question is if you are seeing levels of memory beyond something that you would consider 'normal'?

[lucascebertin]

Sorry for taking so long, @ppekrol. I tried a lot of things but my PODs still get bigger over time. Well, I mean, bigger = 400MB after 2.5k requests that query the database with large objects.

Changing the lifecycle from singleton to scoped changed significally this result... still measuring but it's stay under 170MB, what is really acceptable for me.

I don't know if I should tweak something else to reduce memory and stay singleton, any ideas? BTW, tried reduce de Http Cache Size, same results of 400MB+.

[ppekrol]

DocumentStore should definitely be singleton, but the Session should be scoped. I think that the best way would be to take a memory dump and analyze what is taking memory there exactly. Then we can see what can be done.

[lucascebertin]

Got it, I'll try to do that and get back with something else. Thanks!