search

discord-net / Discord.Net

An unofficial .Net wrapper for the Discord API (https://discord.com/)
https://discordnet.dev
MIT License
3.28k stars 743 forks source link

Discord.Net should not throw unobserved task exceptions #841

Open JustArchi opened 6 years ago

JustArchi commented 6 years ago 
ArchiBoT|OnUnobservedTaskException() System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (WebSocket connection was closed) ---> System.Exception: WebSocket connection was closed ---> System.Net.WebSockets.WebSocketException: The remote party closed the WebSocket connection without completing the close handshake. ---> System.Net.WebSockets.WebSocketException: An exception caused the WebSocket to enter the Aborted state. Please see the InnerException, if present, for more details.
   at System.Net.WebSockets.ManagedWebSocket.<CloseWithReceiveErrorAndThrowAsync>d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Net.WebSockets.ManagedWebSocket.<ReceiveAsyncPrivate>d__61.MoveNext()
   --- End of inner exception stack trace ---
   at System.Net.WebSockets.ManagedWebSocket.<ReceiveAsyncPrivate>d__61.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.Net.WebSockets.DefaultWebSocketClient.<RunAsync>d__32.MoveNext()
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.ConnectionManager.<WaitAsync>d__33.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.WebSocket.DiscordSocketClient.<OnConnectingAsync>d__93.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Discord.WebSocket.DiscordSocketClient.<OnConnectingAsync>d__93.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.ConnectionManager.<ConnectAsync>d__30.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.ConnectionManager.<>c__DisplayClass28_0.<<StartAsync>b__0>d.MoveNext()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.Exception: WebSocket connection was closed ---> System.Net.WebSockets.WebSocketException: The remote party closed the WebSocket connection without completing the close handshake. ---> System.Net.WebSockets.WebSocketException: An exception caused the WebSocket to enter the Aborted state. Please see the InnerException, if present, for more details.
   at System.Net.WebSockets.ManagedWebSocket.<CloseWithReceiveErrorAndThrowAsync>d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Net.WebSockets.ManagedWebSocket.<ReceiveAsyncPrivate>d__61.MoveNext()
   --- End of inner exception stack trace ---
   at System.Net.WebSockets.ManagedWebSocket.<ReceiveAsyncPrivate>d__61.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.Net.WebSockets.DefaultWebSocketClient.<RunAsync>d__32.MoveNext()
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.ConnectionManager.<WaitAsync>d__33.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.WebSocket.DiscordSocketClient.<OnConnectingAsync>d__93.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Discord.WebSocket.DiscordSocketClient.<OnConnectingAsync>d__93.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.ConnectionManager.<ConnectAsync>d__30.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.ConnectionManager.<>c__DisplayClass28_0.<<StartAsync>b__0>d.MoveNext()<---

This was on latest 1.0.2.

foxbot commented 6 years ago

I've never seen this one before, can you provide any additional details about the environment you're running in?

OS, Architecture, .NET Framework or Core + Version, WebSocket Provider?

JustArchi commented 6 years ago

Sure thing. .NET Core 2.0 (netcoreapp2.0) running on Debian Stretch, I'm using only Discord.Net.Websocket package in version 1.0.2 (so also all its dependencies). This exception pops up mainly on Discord network disconnect (e.g. when Cloudflare proxy is restarting). Normally it's not seen as you need to subscribe to TaskScheduler.UnobservedTaskException event in order to see it. In general cause for this is library not observing some faulted task called by itself.

Thank you in advance.

JustArchi commented 6 years ago

Still getting a lot of these basically in any timeout situation.

2017-11-16 02:13:38|FATAL|ArchiBoT|OnUnobservedTaskException() System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The operation has timed out.) ---> System.TimeoutException: The operation has timed out.
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.ConnectionManager.<WaitAsync>d__33.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.WebSocket.DiscordSocketClient.<OnConnectingAsync>d__93.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Discord.WebSocket.DiscordSocketClient.<OnConnectingAsync>d__93.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.ConnectionManager.<ConnectAsync>d__30.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.ConnectionManager.<>c__DisplayClass28_0.<<StartAsync>b__0>d.MoveNext()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.TimeoutException: The operation has timed out.
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.ConnectionManager.<WaitAsync>d__33.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.WebSocket.DiscordSocketClient.<OnConnectingAsync>d__93.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Discord.WebSocket.DiscordSocketClient.<OnConnectingAsync>d__93.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.ConnectionManager.<ConnectAsync>d__30.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.ConnectionManager.<>c__DisplayClass28_0.<<StartAsync>b__0>d.MoveNext()<---
khionu commented 6 years ago

Can you try creating a bare bot in this environment? No other dependencies, no other packages, just Discord.Net.Websocket and the bare minimum code to get it to connect. Just to rule out any possible interference.

Also, if you can provide the full output of dotnet --info, that would be helpful.

JustArchi commented 6 years ago

There is no possible interference - Discord.Net is not observing its own tasks, therefore faulted tasks are being thrown back in TaskScheduler.UnobservedTaskException, I'm not sure what other culprit you're looking for here. This is totally unrelated to anything that can work by side, nothing is touching Discord.Net internals, as you can see in the stacktrace.

Sure I can create minimal repro for this artificially, but you can as well use any Dicord.Net bot connecting example, and just add in Main() method before initialization something like:

TaskScheduler.UnobservedTaskException += (s, e) => Console.WriteLine(e.Exception);

Then let bot connect, unplug your network connection and see what's happening.

$ dotnet --info
.NET Command Line Tools (2.0.2)

Product Information:
 Version:            2.0.2
 Commit SHA-1 hash:  a04b4bf512

Runtime Environment:
 OS Name:     debian
 OS Version:  
 OS Platform: Linux
 RID:         debian-x64
 Base Path:   /opt/dotnet/sdk/2.0.2/

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.0
  Build    : e8b8861ac7faf042c87a5c2f9f2d04c98b69f28d
khionu commented 6 years ago

Humor me, please. I've seen a lot of crazy things in my time.

You're the first person to be reporting this issue, and it's something that you would think would affect many people. For that reason, we need to be 100% certain that it's not just your end.

JustArchi commented 6 years ago

You're the first person to be reporting this issue, and it's something that you would think would affect many people. For that reason, we need to be 100% certain that it's not just your end.

Because it's not a normal thing for everybody to subscribe to task scheduler in order to catch unobserved task exceptions that are used mainly for debugging purposes with catching third-party library leftovers (like this one), and that I also report in corefx for things I stumble upon. I could just ignore all unobserved task exceptions in Discord.Net, close this issue and call it a day rather than spending time trying to point out potential problems with the library in order to improve its own code. I'm not sure if you have the slightest idea what I'm reporting here, or just following some script that assumes it's user's fault regardless, even when he's reporting issues in internal library code and not on a surface that is normally touched by programmer. Just let somebody close the issue if you consider it irrelevant 😕

khionu commented 6 years ago

I'll investigate. From what I've just researched, this would indicate underlying issues. I'll own this issue.

FiniteReality commented 6 years ago

Could you try checking out this branch to see if that resolves the issue?

JustArchi commented 6 years ago

It should address that timeout one, but there's still a plenty of other places with unobserved tasks, let me take a look at my logs to see what else should be reported...

2017-11-09 08:12:38|FATAL|ArchiBoT|OnUnobservedTaskException() System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (An error occurred while sending the request.) ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.CurlException: Couldn't resolve host name
   at System.Net.Http.CurlHandler.ThrowIfCURLEError(CURLcode error)
   at System.Net.Http.CurlHandler.MultiAgent.FinishRequest(StrongToWeakReference`1 easyWrapper, CURLcode messageResult)
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.Net.Rest.DefaultRestClient.<SendInternalAsync>d__14.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
2017-11-16 02:13:38|FATAL|ArchiBoT|OnUnobservedTaskException() System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The operation has timed out.) ---> System.TimeoutException: The operation has timed out.
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.ConnectionManager.<WaitAsync>d__33.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
ArchiBoT|OnUnobservedTaskException() System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (WebSocket connection was closed) ---> System.Exception: WebSocket connection was closed ---> System.Net.WebSockets.WebSocketException: The remote party closed the WebSocket connection without completing the close handshake. ---> System.Net.WebSockets.WebSocketException: An exception caused the WebSocket to enter the Aborted state. Please see the InnerException, if present, for more details.
   at System.Net.WebSockets.ManagedWebSocket.<CloseWithReceiveErrorAndThrowAsync>d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Net.WebSockets.ManagedWebSocket.<ReceiveAsyncPrivate>d__61.MoveNext()
   --- End of inner exception stack trace ---
   at System.Net.WebSockets.ManagedWebSocket.<ReceiveAsyncPrivate>d__61.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Discord.Net.WebSockets.DefaultWebSocketClient.<RunAsync>d__32.MoveNext()
   --- End of inner exception stack trace ---

For now it looks like those are all I managed to get myself - SendInternalAsync(), WaitAsync() and RunAsync() that you've just patched, so 2 out of 3 left 👍. I cut the stacktraces from those above to relevant bits so I won't pollute the message.

The fix looks good though, I'd recommend taking a look at those 2 cases left then sending PR for review and merge into dev branch so other people can test too, but if you'd like just my confirmation alone then that's alright as well. I'll wait for those two cases covered then try to reproduce although it'll probably take a longer while since network malfunctions are happening rarily.

JustArchi commented 6 years ago

Not fixed yet, one more to the collection with different stacktrace, on latest 2.0.0-beta version.

OnUnobservedTaskException() System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Unable to connect to the remote server) ---> System.Net.WebSockets.WebSocketException: Unable to connect to the remote server
   at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options)
   at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken)
   at Discord.Net.WebSockets.DefaultWebSocketClient.ConnectInternalAsync(String host)
   at Discord.Net.WebSockets.DefaultWebSocketClient.ConnectAsync(String host)
   at Discord.API.DiscordSocketApiClient.ConnectInternalAsync()
   at Discord.API.DiscordSocketApiClient.ConnectInternalAsync()
   at Discord.API.DiscordSocketApiClient.ConnectAsync()
   at Discord.WebSocket.DiscordSocketClient.OnConnectingAsync()
   at Discord.WebSocket.DiscordSocketClient.OnConnectingAsync()
   at Discord.ConnectionManager.ConnectAsync(CancellationTokenSource reconnectCancelToken)
   at Discord.ConnectionManager.<>c__DisplayClass28_0.<<StartAsync>b__0>d.MoveNext()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.Net.WebSockets.WebSocketException (0x80004005): Unable to connect to the remote server
   at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options)
   at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken)
   at Discord.Net.WebSockets.DefaultWebSocketClient.ConnectInternalAsync(String host)
   at Discord.Net.WebSockets.DefaultWebSocketClient.ConnectAsync(String host)
   at Discord.API.DiscordSocketApiClient.ConnectInternalAsync()
   at Discord.API.DiscordSocketApiClient.ConnectInternalAsync()
   at Discord.API.DiscordSocketApiClient.ConnectAsync()
   at Discord.WebSocket.DiscordSocketClient.OnConnectingAsync()
   at Discord.WebSocket.DiscordSocketClient.OnConnectingAsync()
   at Discord.ConnectionManager.ConnectAsync(CancellationTokenSource reconnectCancelToken)
   at Discord.ConnectionManager.<>c__DisplayClass28_0.<<StartAsync>b__0>d.MoveNext()<---
Patrick-van-Halm commented 6 years ago

Have the same (Timeouts) error on Ubuntu x64 running it with Mono

JustArchi commented 6 years ago

Bump, those are all over the place. It's enough for Discord to return 500 during logging in and this happens:

OnUnobservedTaskException() System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The server responded with error 500: 500: Internal Server Error) ---> Discord.Net.HttpException: The server responded with error 500: 500: Internal Server Error
   at Discord.Net.Queue.RequestBucket.SendAsync(RestRequest request)
   at Discord.Net.Queue.RequestQueue.SendAsync(RestRequest request)
   at Discord.API.DiscordRestApiClient.SendInternalAsync(String method, String endpoint, RestRequest request)
   at Discord.API.DiscordRestApiClient.SendAsync[TResponse](String method, String endpoint, String bucketId, ClientBucketType clientBucket, RequestOptions options)
   at Discord.API.DiscordRestApiClient.GetMyUserAsync(RequestOptions options)
   at Discord.Rest.DiscordRestClient.OnLoginAsync(TokenType tokenType, String token)
   at Discord.Rest.BaseDiscordClient.LoginInternalAsync(TokenType tokenType, String token)
   at Discord.Rest.BaseDiscordClient.LoginInternalAsync(TokenType tokenType, String token)
   at Discord.Rest.BaseDiscordClient.LoginAsync(TokenType tokenType, String token, Boolean validateToken)
   at ArchiBoT.Discord.Start(String token)
   --- End of inner exception stack trace ---
---> (Inner Exception #0) Discord.Net.HttpException: The server responded with error 500: 500: Internal Server Error
   at Discord.Net.Queue.RequestBucket.SendAsync(RestRequest request)
   at Discord.Net.Queue.RequestQueue.SendAsync(RestRequest request)
   at Discord.API.DiscordRestApiClient.SendInternalAsync(String method, String endpoint, RestRequest request)
   at Discord.API.DiscordRestApiClient.SendAsync[TResponse](String method, String endpoint, String bucketId, ClientBucketType clientBucket, RequestOptions options)
   at Discord.API.DiscordRestApiClient.GetMyUserAsync(RequestOptions options)
   at Discord.Rest.DiscordRestClient.OnLoginAsync(TokenType tokenType, String token)
   at Discord.Rest.BaseDiscordClient.LoginInternalAsync(TokenType tokenType, String token)
   at Discord.Rest.BaseDiscordClient.LoginInternalAsync(TokenType tokenType, String token)
   at Discord.Rest.BaseDiscordClient.LoginAsync(TokenType tokenType, String token, Boolean validateToken)
   at ArchiBoT.Discord.Start(String token)<---

It shouldn't be hard to reproduce, just fake 500 response on the lowest HttpClient level.

whaevr commented 5 years ago

Is this still being looked into? It happens pretty frequently for my bot, I have a cron job setup so far that checks my bots log every minute and restarts it if its failed out. The below is a log from one day from midnight to 6pm, happens a few times everyday. Also on Ubuntu x64 here, .netcore 2.2

Stopped Environment is Release 23:01:01 Discord Discord.Net v2.0.0-beta2-00988 (API v6) 23:01:02 Gateway Connecting 23:01:03 Gateway Connected 23:01:03 Gateway Ready 00:02:22 Gateway A MessageReceived handler is blocking the gateway task. Environment is Release 00:04:01 Discord Discord.Net v2.0.0-beta2-00988 (API v6) 00:04:02 Gateway Connecting 00:04:02 Gateway Connected 00:04:03 Gateway Ready 10:51:39 Gateway System.Exception: WebSocket connection was closed ---> System.Net.WebSockets.WebSocketException: The remote party closed the WebSocket connection without completing the close handshake. ---> System.Net.WebSockets.WebSocketException: The remote party closed the WebSocket connection without completing the close handshake. at System.Net.WebSockets.ManagedWebSocket.ThrowIfEOFUnexpected(Boolean throwOnPrematureClosure) at System.Net.WebSockets.ManagedWebSocket.EnsureBufferContainsAsync(Int32 minimumRequiredBytes, CancellationToken cancellationToken, Boolean throwOnPrematureClosure) at System.Net.WebSockets.ManagedWebSocket.ReceiveAsyncPrivate[TWebSocketReceiveResultGetter,TWebSocketReceiveResult](Memory1 payloadBuffer, CancellationToken cancellationToken, TWebSocketReceiveResultGetter resultGetter) --- End of inner exception stack trace --- at System.Net.WebSockets.ManagedWebSocket.ReceiveAsyncPrivate[TWebSocketReceiveResultGetter,TWebSocketReceiveResult](Memory1 payloadBuffer, CancellationToken cancellationToken, TWebSocketReceiveResultGetter resultGetter) at Discord.Net.WebSockets.DefaultWebSocketClient.RunAsync(CancellationToken cancelToken) --- End of inner exception stack trace --- at Discord.ConnectionManager.<>cDisplayClass28_0.<b0>d.MoveNext() 10:51:39 Gateway Disconnecting 10:51:39 Gateway Disconnected 10:51:40 Gateway Connecting 10:51:40 Gateway Connected 10:51:40 Gateway Resumed previous session Environment is Release 10:52:02 Discord Discord.Net v2.0.0-beta2-00988 (API v6) 10:52:03 Gateway Connecting 10:52:03 Gateway Connected 10:52:04 Gateway Ready 11:07:33 Gateway System.Exception: WebSocket connection was closed ---> System.Net.WebSockets.WebSocketException: The remote party closed the WebSocket connection without completing the close handshake. ---> System.Net.WebSockets.WebSocketException: The remote party closed the WebSocket connection without completing the close handshake. at System.Net.WebSockets.ManagedWebSocket.ThrowIfEOFUnexpected(Boolean throwOnPrematureClosure) at System.Net.WebSockets.ManagedWebSocket.EnsureBufferContainsAsync(Int32 minimumRequiredBytes, CancellationToken cancellationToken, Boolean throwOnPrematureClosure) at System.Net.WebSockets.ManagedWebSocket.ReceiveAsyncPrivate[TWebSocketReceiveResultGetter,TWebSocketReceiveResult](Memory1 payloadBuffer, CancellationToken cancellationToken, TWebSocketReceiveResultGetter resultGetter) --- End of inner exception stack trace --- at System.Net.WebSockets.ManagedWebSocket.ReceiveAsyncPrivate[TWebSocketReceiveResultGetter,TWebSocketReceiveResult](Memory1 payloadBuffer, CancellationToken cancellationToken, TWebSocketReceiveResultGetter resultGetter) at Discord.Net.WebSockets.DefaultWebSocketClient.RunAsync(CancellationToken cancelToken) --- End of inner exception stack trace --- at Discord.ConnectionManager.<>cDisplayClass28_0.<b0>d.MoveNext() 11:07:33 Gateway Disconnecting 11:07:33 Gateway Disconnected 11:07:34 Gateway Connecting 11:07:34 Gateway Connected 11:07:34 Gateway Resumed previous session Environment is Release 11:08:01 Discord Discord.Net v2.0.0-beta2-00988 (API v6) 11:08:02 Gateway Connecting 11:08:03 Gateway Connected 11:08:03 Gateway Ready 11:16:17 Gateway A MessageReceived handler is blocking the gateway task. Environment is Release 11:17:01 Discord Discord.Net v2.0.0-beta2-00988 (API v6) 11:17:02 Gateway Connecting 11:17:03 Gateway Connected 11:17:04 Gateway Ready 11:50:59 Gateway System.Exception: WebSocket connection was closed ---> System.Net.WebSockets.WebSocketException: The remote party closed the WebSocket connection without completing the close handshake. ---> System.Net.WebSockets.WebSocketException: The remote party closed the WebSocket connection without completing the close handshake. at System.Net.WebSockets.ManagedWebSocket.ThrowIfEOFUnexpected(Boolean throwOnPrematureClosure) at System.Net.WebSockets.ManagedWebSocket.EnsureBufferContainsAsync(Int32 minimumRequiredBytes, CancellationToken cancellationToken, Boolean throwOnPrematureClosure) at System.Net.WebSockets.ManagedWebSocket.ReceiveAsyncPrivate[TWebSocketReceiveResultGetter,TWebSocketReceiveResult](Memory1 payloadBuffer, CancellationToken cancellationToken, TWebSocketReceiveResultGetter resultGetter) --- End of inner exception stack trace --- at System.Net.WebSockets.ManagedWebSocket.ReceiveAsyncPrivate[TWebSocketReceiveResultGetter,TWebSocketReceiveResult](Memory1 payloadBuffer, CancellationToken cancellationToken, TWebSocketReceiveResultGetter resultGetter) at Discord.Net.WebSockets.DefaultWebSocketClient.RunAsync(CancellationToken cancelToken) --- End of inner exception stack trace --- at Discord.ConnectionManager.<>cDisplayClass28_0.<b0>d.MoveNext() 11:50:59 Gateway Disconnecting 11:50:59 Gateway Disconnected 11:51:00 Gateway Connecting 11:51:00 Gateway Connected 11:51:00 Gateway Resumed previous session Environment is Release 11:51:02 Discord Discord.Net v2.0.0-beta2-00988 (API v6) 11:51:02 Gateway Connecting 11:51:03 Gateway Connected 11:51:04 Gateway Ready 14:05:21 Gateway System.Exception: WebSocket connection was closed ---> Discord.Net.WebSocketClosedException: The server sent close 1000: "" at Discord.Net.WebSockets.DefaultWebSocketClient.RunAsync(CancellationToken cancelToken) --- End of inner exception stack trace --- at Discord.ConnectionManager.<>cDisplayClass28_0.<b0>d.MoveNext() 14:05:21 Gateway Disconnecting 14:05:21 Gateway Disconnected 14:05:22 Gateway Connecting 14:05:23 Gateway Connected 14:05:23 Gateway Resumed previous session Environment is Release 14:07:02 Discord Discord.Net v2.0.0-beta2-00988 (API v6) 14:07:02 Gateway Connecting 14:07:03 Gateway Connected 14:07:04 Gateway Ready

JustArchi commented 5 years ago

One more to the collection, from v2.0.1:

ArchiBoT|OnUnobservedTaskException() Discord.Net.HttpException: The server responded with error 500: 500: Internal Server Error
at Discord.Net.Queue.RequestBucket.SendAsync(RestRequest request)
at Discord.Net.Queue.RequestQueue.SendAsync(RestRequest request)
at Discord.API.DiscordRestApiClient.SendInternalAsync(String method, String endpoint, RestRequest request)
at Discord.API.DiscordRestApiClient.SendAsync[TResponse](String method, String endpoint, String bucketId, ClientBucketType clientBucket, RequestOptions options)
at Discord.API.DiscordRestApiClient.GetMyUserAsync(RequestOptions options)
at Discord.Rest.DiscordRestClient.OnLoginAsync(TokenType tokenType, String token)
at Discord.Rest.BaseDiscordClient.LoginInternalAsync(TokenType tokenType, String token, Boolean validateToken)
at Discord.Rest.BaseDiscordClient.LoginInternalAsync(TokenType tokenType, String token, Boolean validateToken)
at Discord.Rest.BaseDiscordClient.LoginAsync(TokenType tokenType, String token, Boolean validateToken)
at ArchiBoT.Chat.Discord.Core.Start(String token)

This is the same one as reported above by me.

JustArchi commented 4 years ago

Still a lot of those everywhere. That was on 2.2.0.

ArchiBot|OnUnobservedTaskException() System.TimeoutException: The operation has timed out.
at Discord.ConnectionManager.WaitAsync()
at Discord.WebSocket.DiscordSocketClient.OnConnectingAsync()
at Discord.WebSocket.DiscordSocketClient.OnConnectingAsync()
at Discord.ConnectionManager.ConnectAsync(CancellationTokenSource reconnectCancelToken)
at Discord.ConnectionManager.<>c__DisplayClass29_0.<<StartAsync>b__0>d.MoveNext()
ArchiBot|OnUnobservedTaskException() System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
---> System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer.
---> System.Net.Sockets.SocketException (104): Connection reset by peer
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request)
at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at Discord.Net.Rest.DefaultRestClient.SendInternalAsync(HttpRequestMessage request, CancellationToken cancelToken, Boolean headerOnly)
at Discord.Net.Rest.DefaultRestClient.SendAsync(String method, String endpoint, CancellationToken cancelToken, Boolean headerOnly, String reason)
at Discord.Net.Queue.RestRequest.SendAsync()
at Discord.Net.Queue.RequestBucket.SendAsync(RestRequest request)
at Discord.Net.Queue.RequestQueue.SendAsync(RestRequest request)
at Discord.API.DiscordRestApiClient.SendInternalAsync(String method, String endpoint, RestRequest request)
at Discord.API.DiscordRestApiClient.SendAsync[TResponse](String method, String endpoint, String bucketId, ClientBucketType clientBucket, RequestOptions options)
at Discord.API.DiscordRestApiClient.GetGatewayAsync(RequestOptions options)
at Discord.API.DiscordSocketApiClient.ConnectInternalAsync()
at Discord.API.DiscordSocketApiClient.ConnectInternalAsync()
at Discord.API.DiscordSocketApiClient.ConnectAsync()
at Discord.WebSocket.DiscordSocketClient.OnConnectingAsync()
at Discord.WebSocket.DiscordSocketClient.OnConnectingAsync()
at Discord.ConnectionManager.ConnectAsync(CancellationTokenSource reconnectCancelToken)
at Discord.ConnectionManager.<>c__DisplayClass29_0.<<StartAsync>b__0>d.MoveNext()
Crytilis commented 2 years ago

I also get these, trying to wrap my head around it.

Discord.WebSocket.DiscordSocketClient[0] Gateway: System.Exception: WebSocket connection was closed ---> System.Net.WebSockets.WebSocketException (0x80004005): The remote party closed the WebSocket connection without completing the close handshake. at System.Net.WebSockets.ManagedWebSocket.ThrowIfEOFUnexpected(Boolean throwOnPrematureClosure) at System.Net.WebSockets.ManagedWebSocket.EnsureBufferContainsAsync(Int32 minimumRequiredBytes, CancellationToken cancellationToken, Boolean throwOnPrematureClosure) at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder1.StateMachineBox1.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.WebSockets.ManagedWebSocket.ReceiveAsyncPrivate[TResult](Memory1 payloadBuffer, CancellationToken cancellationToken) at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder1.StateMachineBox1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token) at System.Threading.Tasks.ValueTask1.ValueTaskSourceAsTask.<>c.<.cctor>b4_0(Object state) --- End of stack trace from previous location --- at Discord.Net.WebSockets.DefaultWebSocketClient.RunAsync(CancellationToken cancelToken) --- End of inner exception stack trace --- at Discord.ConnectionManager.<>c__DisplayClass29_0.<b0>d.MoveNext()

quinchs commented 2 years ago

Does this still occur in V3?

JustArchi commented 2 years ago

I've removed my existing ignores for those unobserved task exceptions since there was no end to them, and I'll see if I receive any when Discord servers once again will have issues ensuring stable connection.

But there was so many of them I heavily doubt all of them disappeared from unrelated code changes alone.

JustArchi commented 2 years ago

@quinchs got one:

ArchiBot|OnUnobservedTaskException() System.TimeoutException: The operation has timed out.
at Discord.ConnectionManager.WaitAsync()
at Discord.WebSocket.DiscordSocketClient.OnConnectingAsync()
at Discord.ConnectionManager.ConnectAsync(CancellationTokenSource reconnectCancelToken)
at Discord.ConnectionManager.<>c__DisplayClass29_0.<<StartAsync>b__0>d.MoveNext()

Discord.Net 3.3.0 from nuget, .NET 6.0.

JustArchi commented 2 years ago

Just to clear that up so we're on the same page:

Unobserved task exceptions can be subscribed to through TaskScheduler.UnobservedTaskException event. It's very useful for debugging, as it ensures that we don't have faulting tasks in the background, which may or may not affect the rest of the process, in particular, various soft and hard deadlocks as well as stuff that "should not happen".

More can be read on https://docs.microsoft.com/dotnet/api/system.threading.tasks.taskscheduler.unobservedtaskexception?view=net-6.0

It's a good practice to address every unobserved task exception. If by any chance it's determined that it's harmless for the rest of the execution, the code responsible for calling it should handle it without relying on GC picking it up, cluttering the event in the process. Sometimes however it might indicate that indeed something unexpected happens which may require code logic changes to take that possibility into account - this is very often cause for hard to diagnose issues, as it's not apparent some task we expect to finish gets exception-cancelled in the middle.

See my other issues in the same manner if you want references on how one can deal with it: https://github.com/dotnet/runtime/issues/25838 https://github.com/dotnet/runtime/issues/25371 https://github.com/dotnet/runtime/issues/23539

quinchs commented 2 years ago

Il look into this

JustArchi commented 2 years ago

@quinchs another to the collection:

2022-02-17 09:11:00|FATAL|ArchiBot|OnUnobservedTaskException() System.Net.Http.HttpRequestException: An error occurred while sending the request.
 ---> System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer.
 ---> System.Net.Sockets.SocketException (104): Connection reset by peer
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.CreateException(SocketError error, Boolean forAsyncThrow)
   at System.Net.Sockets.NetworkStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread)
   at System.Net.Sockets.SocketAsyncEventArgs.FinishOperationAsyncSuccess(Int32 bytesTransferred, SocketFlags flags)
   at System.Net.Sockets.SocketAsyncEngine.System.Threading.IThreadPoolWorkItem.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
--- End of stack trace from previous location ---

   --- End of inner exception stack trace ---
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
   at System.Net.Http.HttpConnection.<CheckUsabilityOnScavenge>g__ReadAheadWithZeroByteReadAsync|44_0()
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Discord.Net.Rest.DefaultRestClient.SendInternalAsync(HttpRequestMessage request, CancellationToken cancelToken, Boolean headerOnly)
   at Discord.Net.Rest.DefaultRestClient.SendAsync(String method, String endpoint, CancellationToken cancelToken, Boolean headerOnly, String reason)
   at Discord.Net.Queue.RestRequest.SendAsync()
   at Discord.Net.Queue.RequestBucket.SendAsync(RestRequest request)
   at Discord.Net.Queue.RequestQueue.SendAsync(RestRequest request)
   at Discord.API.DiscordRestApiClient.SendInternalAsync(String method, String endpoint, RestRequest request)
   at Discord.API.DiscordRestApiClient.SendAsync[TResponse](String method, String endpoint, BucketId bucketId, ClientBucketType clientBucket, RequestOptions options)
   at Discord.API.DiscordRestApiClient.GetGatewayAsync(RequestOptions options)
   at Discord.API.DiscordSocketApiClient.ConnectInternalAsync()
   at Discord.API.DiscordSocketApiClient.ConnectInternalAsync()
   at Discord.API.DiscordSocketApiClient.ConnectAsync()
   at Discord.WebSocket.DiscordSocketClient.OnConnectingAsync()
   at Discord.ConnectionManager.ConnectAsync(CancellationTokenSource reconnectCancelToken)
   at Discord.ConnectionManager.<>c__DisplayClass29_0.<<StartAsync>b__0>d.MoveNext()

3.3.0 for now, I'll update to latest 3.3.2 soon - but I doubt you've fixed that already.