tihomir-kit / InfluxData.Net

InfluxData TICK stack .net library.
MIT License
156 stars 52 forks source link

Need Help: Unable to read data from the transport connection: The connection was closed. #81

Open ReactRS opened 5 years ago

ReactRS commented 5 years ago

Hi,

We are using the library InfluxDB.Net Release 8.0.1 to read data from Influx DB hosted in the cloud. We have 1-minute interval data stored in the cloud database. We have a requirement to generate .csv report based on user request. The REST call works seamlessly for MultiQueryAsync (190 Queries, each query reading different Data Point) for three weeks worth of data. i.e 190 Columns 1446 Rows per day 21 = 5,769,540 data points.

However when we try to read a month worth of Data: the Influx DB call fails with the below exception InnerException = {"Unable to read data from the transport connection: The connection was closed."}

at System.Net.ConnectStream.EndRead(IAsyncResult asyncResult) at System.IO.Stream.<>c.b__43_1(Stream stream, IAsyncResult asyncResult) at System.Threading.Tasks.TaskFactory1.FromAsyncTrimPromise1.Complete(TInstance thisRef, Func`3 endMethod, IAsyncResult asyncResult, Boolean requiresSynchronization) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.d4.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Net.Http.StreamToStreamCopy.d3.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Net.Http.HttpContent.d__49.MoveNext()

The Client Request is handled by WebAPI2.0, below is the code snippet.

image

To avoid reading larger data we explored the QuerySync/MultiQyery Async with chunk Size of 2000, this fails with

Exception of type 'System.OutOfMemoryException' was thrown.

at System.String.SplitInternal(Char[] separator, Int32 count, StringSplitOptions options) at InfluxData.Net.InfluxDb.ClientModules.ClientModuleBase.SplitChunkedResponse(IInfluxDataApiResponse response) at InfluxData.Net.InfluxDb.ClientModules.ClientModuleBase.ResolveGetSeriesResultChunked(IInfluxDataApiResponse response) at InfluxData.Net.InfluxDb.ClientModules.ClientModuleBase.d13.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at InfluxData.Net.InfluxDb.ClientModules.BasicClientModule.d2.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult() at Condor.Web.Controllers.API.ReportController.d__3.MoveNext() in

If anyone has experience resolving this issue please share your inputs.