splunk / splunk-sdk-csharp-pcl

Splunk's next generation C# SDK
https://dev.splunk.com/enterprise/docs/csharp
Apache License 2.0
64 stars 46 forks source link

Splunk Client is missing an implementation of IDataReader to consume from SearchResultStream #109

Closed YulerB closed 2 years ago

YulerB commented 3 years ago

As a consumer of your Client API.

I want to load data into SqlServer using Bulk Copy

Therefore I need a IDataReader instead of SearchResultStream.

Please add https://github.com/YulerB/SplunkClientDataReader/blob/main/SearchResultStreamDataReader.cs

So that clients can execute SQL-Server database loading in streaming mode using 1 implementation:

  using(SqlBulkCopy sqlBulkCopy = new SqlBulkCopy(connectionString, SqlBulkCopyOptions.Default)){
    sqlBulkCopy.BatchSize = 5000;
    sqlBulkCopy.EnableStreaming = true;
    sqlBulkCopy.BulkCopyTimeout = 360;
    sqlBulkCopy.DestinationTableName = "dbo.SplunkData";

    using (var context = new Context(Scheme.Https, endPoint.Host, endPoint.Port))
    {
      using (var client = new Service(context))
      {
        await client.LogOnAsync(credential.UserName, credential.Password).ConfigureAwait(false);

        using(var results = await client.ExportSearchResultsAsync(search, searchExportArgs).ConfigureAwait(false))
        {
          using(var reader = new SearchResultStreamDataReader(results))
          {
            await sqlBlockCopy.WriteToServerAsync(reader).ConfigureAwait(false);
          }
        }
      }
    }
  }
XtremeOwnageDotCom commented 2 years ago

Just a heads up... Splunk decided to depreciate this repository, its noted on at the top of the readme in the root of the repo.

If you are wanting to continue using it, I would suggest forking it, and making the changes yourself.

ncanumalla-splunk commented 2 years ago

This SDK is deprecated and no longer under active development.