search

akkadotnet / akka.net

Canonical actor model implementation for .NET with local + distributed actors in C# and F#.
http://getakka.net
Other
4.7k stars 1.04k forks source link

System.Net.Sockets.SocketException during coordinated shutdown CLR termination hook #3208

Open gengle opened 6 years ago

gengle commented 6 years ago

I sporadically receive the following exception when node (role=client) disconnects from cluster.

[08:45:33 INF] Starting coordinated shutdown from CLR termination hook. [08:45:33 INF] Shutting down remote daemon. [08:45:33 INF] Remote daemon shut down; proceeding with flushing remote transports.

System.Net.Sockets.SocketException (0x80004005): An established connection was aborted by the software in your host machine at DotNetty.Transport.Channels.Sockets.TcpSocketChannel.DoReadBytes(IByteBuffer byteBuf) at DotNetty.Transport.Channels.Sockets.AbstractSocketByteChannel.SocketByteChannelUnsafe.FinishRead(SocketChannelAsyncOperation operation) [08:45:33 INF] Remoting shut down. `

The following packages are used on net461 `

`

This error occurs randomly but I have noticed the frequency increase when spawn multiple instances node are spawned on the same machine (with same IP and dynamic port#)

I'll update this issue once I've isolated a consistent pattern we can reproduce.

Aaronontheweb commented 6 years ago

@gengle not sure if this is worth fixing, but it appears to be an issue when we terminate DotNetty and a particular connection doesn't get a chance to shut down before the timeout threshold and we abort it (we force the shutdown to occur one way or another.) We can look into it but it's usually harmless.

maxcherednik commented 6 years ago

the shutdown is not graceful then :) (the fan of clear log here)

gengle commented 6 years ago

@Aaronontheweb thanks for the feedback. In this situation, the node is a client tool which quickly connects and disconnects from the cluster. Maybe I should consider using ClusterClient instead in this situation.

maxcherednik commented 6 years ago

hey @Aaronontheweb, shall I have a look the remoting part for a graceful shutdown?