datalust / serilog-sinks-seq

A Serilog sink that writes events to the Seq structured log server
https://datalust.co/seq
Apache License 2.0
239 stars 50 forks source link

System.AggregateException being thrown when handling TaskScheduler.UnobservedTaskException events in WPF application #205

Closed TheNr1Guest closed 10 months ago

TheNr1Guest commented 11 months ago

In our WPF application we listen to TaskScheduler.UnobservedTaskException for any exceptions:

TaskScheduler.UnobservedTaskException += (_, e) => logger.LogError(e.Exception, "An unobserved task exception occurred!");

We see a lot of the following exceptions being logged:

System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Unable to read data from the transport connection: De externe host heeft een verbinding verbroken..)
 ---> System.IO.IOException: Unable to read data from the transport connection: De externe host heeft een verbinding verbroken..
 ---> System.Net.Sockets.SocketException (10054): De externe host heeft een verbinding verbroken.
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.CreateException(SocketError error, Boolean forAsyncThrow)
   at System.Net.Sockets.NetworkStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnection.PrepareForReuse(Boolean async)
   at System.Net.Http.HttpConnectionPool.TryGetPooledHttp11Connection(HttpRequestMessage request, Boolean async, HttpConnection& connection, HttpConnectionWaiter`1& waiter)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Net.Http.HttpConnectionPool.SendAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPoolManager.SendAsyncCore(HttpRequestMessage request, Uri proxyUri, Boolean async, Boolean doRequestAuth, Boolean isProxyConnect, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPoolManager.SendAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpMessageHandlerStage.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.SocketsHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Net.Http.HttpClient.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at Serilog.Sinks.Seq.Http.SeqIngestionApiClient.TryIngestAsync(String payload, String mediaType)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at Serilog.Sinks.Seq.Http.SeqIngestionApi.IngestAsync(String clefPayload)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at Serilog.Sinks.Seq.Http.SeqIngestionApi.IngestAsync(String clefPayload)
   at Serilog.Sinks.Seq.Batched.BatchedSeqSink.EmitBatchAsync(IEnumerable`1 events)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at Serilog.Sinks.Seq.Batched.BatchedSeqSink.EmitBatchAsync(IEnumerable`1 events)
   at Serilog.Sinks.PeriodicBatching.PeriodicBatchingSink.OnTick()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at Serilog.Sinks.PeriodicBatching.PortableTimer.OnTick()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Threading.TimerQueueTimer.Fire(Boolean isThreadPool)
   at System.Threading.TimerQueue.FireNextTimers()
   at System.Threading.TimerQueue.System.Threading.IThreadPoolWorkItem.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback()
--- End of stack trace from previous location ---

   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---

This looks to be an issue with Task error handling within this library as far as I can tell.

nblumhardt commented 11 months ago

That's interesting - thanks for the heads-up, @TheNr1Guest.

My guess is that the issue will be related to Serilog.Sinks.PeriodicBatching, but I'll dig in and figure out what might be going on :+1: