Timeout exception on job.GetSearchResultsAsync

[grbinho]

Hi,

is it possible to set a timeout for job.GetSearchResultAsync ? I was able to set the timeout on the context for the service, but that does not seem to have an effect on this call. I'm getting this exception

System.Threading.Tasks.TaskCanceledException occurred
  HResult=0x8013153B
  Message=A task was canceled.
  Source=<Cannot evaluate the exception source>
  StackTrace:
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Splunk.Client.Context.<SendAsync>d__37.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Splunk.Client.Context.<GetResponseAsync>d__36.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Splunk.Client.Context.<GetAsync>d__23.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Splunk.Client.Job.<GetAsync>d__111.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Splunk.Client.Job.<TransitionAsync>d__112.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Splunk.Client.Job.<GetSearchResultsAsync>d__142.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Splunk.Client.Job.<GetSearchResultsAsync>d__120.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Splunk.Client.Job.<GetSearchResultsAsync>d__121.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()

On the server side, i run the same query and it is done withing a minute. It also returns 4MB of data, so I don't see why this is an issue.

It looks to me like there is a fixed timeout of 30 seconds somewhere?

Is that the case, or am I going at it wrong?

[tdhellmann]

Hi @grbinho - can you give us a code snippet demonstrating this issue to help us look into it? Also, anything else you can tell us about your deployment (which version of Splunk; is it a clustered deployment; is it on Splunk Cloud; etc.) could be helpful.

[grbinho]

Sure,

We are using spunk enterprise, build: 6.5.2. To my knowledge this is a clustered/Load balanced setup. I'm not completely sure about the details, but I can follow up with the guys that are maintaining it.

And here is the code

public async Task QuerySplunk(string query, Action<SearchResult> resultFiller)
{
    try
    {
        using (var httpClientHandler = new WebRequestHandler())
        {
            httpClientHandler.CachePolicy = new RequestCachePolicy(RequestCacheLevel.NoCacheNoStore);

            using (var context = new Context(Scheme.Https, _configuration.SplunkHost, _configuration.SplunkPort, TimeSpan.FromMinutes(5), httpClientHandler))
            {
                using (var service = new Service(context))
                {
                    try
                    {
                        await service.LogOnAsync(_configuration.SplunkUsername, _configuration.SplunkPassword);
                    }
                    catch (AuthenticationFailureException authException)
                    {
                        _logger.Error("Splunk server: Login error", authException);
                        throw;
                    }

                    var job = await service.Jobs.CreateAsync(query);

                    using (var stream = await job.GetSearchResultsAsync())
                    {
                        foreach (var item in stream)
                        {
                            resultFiller(item);
                        }
                    }

                    await job.RemoveAsync();
                }
            }
        }
    }
    catch (Exception ex)
    {
        _logger.Error("Splunk server: query management error", ex);
        throw;
    }
}

[ncanumalla-splunk]

This SDK is deprecated and no longer under active development.