Memory leak with Expando object

[grbinho]

Hi,

We are using Splunk.Client to export data from our Splunk instalation. At this moment we are querying our instance every minute for the previous minute of data. We are running a "Normal" search as demonstrated in you documentation.

It looks like we are running into a memory leak when doing this.

Our result set contains 12 columns and between 20000 - 50000 records per query.

Here is the code we run:

try
{
    using (var service = new Service(Scheme.Https, _configuration.SplunkHost, _configuration.SplunkPort))
    {
        try
        {
            await service.LogOnAsync(_configuration.SplunkUsername, _configuration.SplunkPassword);
        }
        catch (AuthenticationFailureException authException)
        {
            _logger.Error("Splunk server: Login error", authException);
            throw;
        }

        var job = await service.Jobs.CreateAsync(query, mode: ExecutionMode.Normal);

        using (var stream = await job.GetSearchResultsAsync())
        {
            foreach (var result in stream)
            {
                var row = resultTransformer(result);
                tableResult.Rows.Add(row);
            }
        }

        return tableResult;
    }
}
catch (Exception ex)
{
    _logger.Error("Splunk server: query management error", ex);
    throw;
}

This code is running in a loop. We are seeing a big number of objects being created, and not collected by GC (even when forcing on all generations).

While investigating we ran into this .NET bug report

The pattern used in your SearchResult class looks to be exactly what that bug report is explaining.

Do you have any insight that might help us? Maybe an alternative way to do the query that does not use SearchResult, but gets raw string data (in CSV or some other format)?

[faisalmansoor]

+1 I am running into this also.

I am not sure, but, it might be caused by how dynamic calls are cached by c# compiler.

https://github.com/dotnet/corefx/pull/15397 https://github.com/dotnet/corefx/issues/11934 https://stackoverflow.com/questions/41354549/memory-leak-when-using-callsite

[ncanumalla-splunk]

This SDK is deprecated and no longer under active development.