fatal exception

[Jakobi-mirsk]

hi today we have had a breakdown leaving us with the following unhandled exception in the event viewer: _Application: Mirsk.RSR.IBM.exe Framework Version: v4.0.30319 Description: The process was terminated due to an unhandled exception. Exception Info: System.ObjectDisposedException at System.Net.WebSockets.ClientWebSocket.ConnectAsync(System.Uri, System.Threading.CancellationToken) at SocketIOClient.Transport.DefaultClientWebSocket+d10.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task) at SocketIOClient.Transport.WebSocketTransport+d33.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task) at SocketIOClient.Routers.WebSocketRouter+d5.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task) at SocketIOClient.SocketIO+d81.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at SocketIOClient.SocketIO+d81.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task) at SocketIOClient.SocketIO+d105.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean) at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean) at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem() at System.Threading.ThreadPoolWorkQueue.Dispatch()

_ I don't know how to handle this exception. I guess it is thrown in the ConnectAsync task. Why is it thrown and how do I prevent it from tearing down the hole program?

[Jakobi-mirsk]

here is some extra information. the scenario is that we have about 45 connections that all for some reason fails to transmit data. This can very well be because of the other end having problems, but i have seen that when the EmitAsync task fails (IsFaulted) then the socket does not reconnect. So i do it manually in this case. So all of these connections are tried reconnected manually again, and it is in that scenario that we see this problem. This might by my own fault, but i really would like to know how to prevent my program from terminating like that. The weird thing is that i do have a try catch around the ConnectAsync call, so my guess is that this is happening inside the lib in a thread.

[Jakobi-mirsk]

I found a weird thing happening just before the for the crash. This is my own log but it gives an idea about what is happening: SocketIO (id:1279997 socketid:): Connecting SocketIO (id:1279997 socketid:wogzdUCwxGVgZyazAAI): Connected event received SocketIO (id:1279997 socketid:wogzdUCwxGVgZyazAAI): Disconnecting. (34 slow transmissions) SocketIO (id:1279997 socketid:wogzdUCwxGVgZyazAAI): Failed to disconnect within specified timeout SocketIO (id:1279997 socketid:wogzdUCwxGVgZyazAAI): Failed to disconnect normally SocketIO (id:1279997 socketid:wogzdUCwxGVgZyazAAI_): Disconnected event received: transport close SocketIO (id:1279997 socketid:Unknown): Failed to send, task faulted SocketIO (id:1279997 socketid:Unknown): Connected event received.

so what is happening is that i make a connection successfully then i send data(cannot be seen from this trace) and at some point i try to disconnect, but it fails to do so within 5 sec. so i dispose the connection(also cannot be seen in this trace), after i dispose the connection an emitasync is failing with task.IsFaulted = true. after that a connected event is fired from the lib.

[Jakobi-mirsk]

I was looking through your code and found something weird that might explain the reason why it tries to reconnect after I try to disconnect. In the SocketIO class in method InvokeDisconnect you validate a disconnect reason. In that validation you validate DisconnectReason.IOServerDisconnect two times, shouldn't the second time be replaced with another validation reason?

[doghappy]

Yes, my mistake. We should use IOClientDisconnect instead of the second one.

[doghappy]

It seems that this problem is a bit complicated, can it be reproduced?

[Jakobi-mirsk]

Yes, it is very complicated :-D I have another reported issue that talks about delays when calling EmitAsync. I think this is a result of that problem. before this unhandled exception crashed my program I had allot of slow transmissions. I am not sure how the program can end up in this state. It can be the server that has a hard time accepting the transmissions or it might be the machine that runs my program that is exhausted in some unapparent way (it does not seen to be CPU or Memory related). So there is really two issues that i am struggling with. Why is transmissions getting delayed and how can an exception in your lib crash my program.

[Jakobi-mirsk]

Another thing is that I think I have found a way to simulate that a thread can crash a program even though a try-catch has been used around the async. call. I think, it might be a god idea to consider using try-catch around calls to the ClientWebSocket object. I fear that exceptions in these methods will result in a program crash. Let me know what you think.

[Jakobi-mirsk]

We just had another brake down. This time the event viewer revealed the following exception: _Application: Mirsk.RSR.IBM.exe Framework Version: v4.0.30319 Description: The process was terminated due to an unhandled exception. Exception Info: System.ObjectDisposedException at System.Threading.CancellationTokenSource.ThrowObjectDisposedException() at System.Net.WebSockets.ClientWebSocket.CreateAndConfigureRequest(System.Uri) at System.Net.WebSockets.ClientWebSocket+d21.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task) at SocketIOClient.Transport.DefaultClientWebSocket+d10.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task) at SocketIOClient.Transport.WebSocketTransport+d33.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task) at SocketIOClient.Routers.WebSocketRouter+d5.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task) at SocketIOClient.SocketIO+d81.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at SocketIOClient.SocketIO+d81.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task) at SocketIOClient.SocketIO+d_105.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean) at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean) at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem() at System.Threading.ThreadPoolWorkQueue.Dispatch()

[doghappy]

hi, which version of SocketIOClient are you using?

[Jakobi-mirsk]

the latest version: 3.0.4

[Jakobi-mirsk]

we have just had a crash again. When do you think you will have an update for the IOClientDisconnect bug?

[doghappy]

I'd love to fix it, but I can't reproduce it yet 😞, so I don't know where to start 😨. Can you create a repo and reproduce it?

[Jakobi-mirsk]

I think that the crash is a direct result of the IOClientDisconnect bug described earlier in this thread, so if you fix that, then I think my problem goes away. I think the reason that we have this problem is that when we disconnect with the IOClientDisconnect reason then because if this code: if (reason != DisconnectReason.IOServerDisconnect && reason != DisconnectReason.IOServerDisconnect) { //In the this cases (explicit disconnection), the client will not try to reconnect and you need to manually call socket.connect(). if (Options.Reconnection) { await ConnectAsync().ConfigureAwait(false); } } the connection is being established again, thought it shouldn't. After we call disconnect we also call dispose and I think this combination is fatal. Not sure if it makes sense.

[doghappy]

has been released

[Jakobi-mirsk]

Great. I am looking forward to test it. Thank you for your effort.

[nsharow]

Hi! The couse for this problem is the code below:

private void StartPolling(CancellationToken cancellationToken)
        {
            Task.Factory.StartNew(async () =>
            {
                while (!cancellationToken.IsCancellationRequested)
                {
                    try
                    {
                        await _httpTransport.GetAsync(_httpUri, CancellationToken.None).ConfigureAwait(false);
                    }
                    catch
                    {
                        OnTransportClosed();
                        throw;
                    }
                }
            }, TaskCreationOptions.LongRunning);
        }

When an exception occurs (ConnectAsync throws it) in the body of an anonymous task, the runtime kills the entire process. We can quickly fix this by disabling the "Reconnection" option

var options = new SocketIOOptions()
          {
              ...
              Reconnection = false,
              ...
          };

or another suitable solution - dispose socket in Disconnected handler. But it would be great if this bug gets fixed

[theolivenbaum]

Guess one alternative would be to pass the exception to the event and just return instead:

try
{
    await _httpTransport.GetAsync(_httpUri, CancellationToken.None).ConfigureAwait(false);
}
catch(Exception E)
{
    OnTransportClosed(E);
    return;
}