OData / odata.net

ODataLib: Open Data Protocol - .NET Libraries and Frameworks
https://docs.microsoft.com/odata
Other
683 stars 349 forks source link

memory occupied and no longer released #3019

Open TZeri opened 1 month ago

TZeri commented 1 month ago

The call to a large odata service takes up a lot of memory and is no longer released.

Assemblies affected

i use this package Version="7.21.3"

Reproduce steps

NAV.NAV nav = new NAV.NAV(new Uri(path)) { Credentials = new NetworkCredential(utente, psw) }; now the memory is image

var listClienti = new List(); var asyncQuery = nav.Clienti as DataServiceQuery; var result = await asyncQuery.ExecuteAsync() as QueryOperationResponse;

image DataServiceQueryContinuation nextLink = null; List toReturn = new List();

do { if (nextLink != null) { result = await nav.ExecuteAsync(nextLink) as QueryOperationResponse; } toReturn.AddRange(result);

} while ((nextLink = result.GetContinuation()) != null);

image

Expected result

memory must go back low

Actual result

heigth usage of memory image

Additional detail

worker service .net 6

habbes commented 1 month ago

@TZeri OData Client has an entity tracker that keeps a copy of entities it has received and some metadata to keep track of local changes so that it can sync with the server when SaveChangesAsync is called. This data remains in memory so long as the DataServiceContext is in memory. If you don't need to persist this data across requests or you don't need to track changer, you could disable entity tracking, this will reduce the amount of data that DataServiceContext keeps around.

using Microsoft.OData.Client;

NAV.NAV nav = new NAV.NAV(new Uri(path))
{
   Credentials = new NetworkCredential(utente, psw),
   MergeOption = MergeOption.NoTracking
}

Alternatively, you could create scoped/short-lived instance of DataServiceContext per request or per series of related requests instead of a singleton or long-lived DataServiceContext. When the DataServiceContext goes out of scope and is eventually garbage collected, the entity tracker and data it holds should also get garbage collected and free up that memory.

Note: we should add this to our docs on perf improvement tips: https://github.com/MicrosoftDocs/OData-docs/issues/317

TZeri commented 1 month ago

Thanks for the response, I have treid with MergeOption.NoTracking but the memory usage not changed. Then i have tried initialize the DataServiceContext in a IDisposable class and the situation non changed. Where I wrong?

habbes commented 1 month ago

@TZeri memory will be held until it's cleaned up by the garbage collection. You can force garbage collection by running GC.Collect(). If memory usage is still high after garbage collection, then you could place a breakpoint in Visual Studio after garbage collection has completed and use the Memory Usage profiler to investigate whether there's something else that's holding a reference to your DataServiceContext or entity tracker or materialized entities and preventing them from getting garbage collected: https://learn.microsoft.com/en-us/visualstudio/profiling/memory-usage?view=vs-2022#managed-types-reports

If we an idea of what's keeping the data in memory after a full garbage collection, it might be easier to diagnose and resolve.