System.InvalidOperationException: The operation is not allowed on non-connected sockets.

planetarium / libplanet

Blockchain in C#/.NET for on-chain, decentralized gaming

GNU Lesser General Public License v2.1

506 stars 142 forks source link

System.InvalidOperationException: The operation is not allowed on non-connected sockets. at System.Net.Sockets.TcpClient.GetStream() at Libplanet.Stun.TurnClient.CreatePermissionAsync(IPEndPoint peerAddress, CancellationToken cancellationToken) in C:\Users\kidon\Desktop\planetarium_repo\NineChronicles.Headless\Lib9c\.Libplanet\Libplanet.Stun\Stun\TurnClient.cs:line 164 at Libplanet.Net.Transports.NetMQTransport.CreatePermission(BoundPeer peer) in C:\Users\kidon\Desktop\planetarium_repo\NineChronicles.Headless\Lib9c\.Libplanet\Libplanet\Net\Transports\NetMQTransport.cs:line 903 at Libplanet.Net.Transports.NetMQTransport.SendMessageWithReplyAsync(BoundPeer peer, Message message, Nullable`1 timeout, Int32 expectedResponses, CancellationToken cancellationToken) in C:\Users\kidon\Desktop\planetarium_repo\NineChronicles.Headless\L ib9c\.Libplanet\Libplanet\Net\Transports\NetMQTransport.cs:line 419 at Libplanet.Net.Transports.NetMQTransport.SendMessageWithReplyAsync(BoundPeer peer, Message message, Nullable`1 timeout, CancellationToken cancellationToken) in C:\Users\kidon\Desktop\planetarium_repo\NineChronicles.Headless\Lib9c\.Libplanet\Libplanet \Net\Transports\NetMQTransport.cs:line 401 at Libplanet.Net.Protocols.KademliaProtocol.PingAsync(BoundPeer target, Nullable`1 timeout, CancellationToken cancellationToken) in C:\Users\kidon\Desktop\planetarium_repo\NineChronicles.Headless\Lib9c\.Libplanet\Libplanet\Net\Protocols\KademliaProtoco l.cs:line 452 at Libplanet.Net.Protocols.KademliaProtocol.ValidateAsync(BoundPeer peer, TimeSpan timeout, CancellationToken cancellationToken) in C:\Users\kidon\Desktop\planetarium_repo\NineChronicles.Headless\Lib9c\.Libplanet\Libplanet\Net\Protocols\KademliaProtoco l.cs:line 533 at Libplanet.Net.Protocols.KademliaProtocol.RefreshTableAsync(TimeSpan maxAge, CancellationToken cancellationToken) in C:\Users\kidon\Desktop\planetarium_repo\NineChronicles.Headless\Lib9c\.Libplanet\Libplanet\Net\Protocols\KademliaProtocol.cs:line 220 at Libplanet.Net.Swarm`1.RefreshTableAsync(TimeSpan period, TimeSpan maxAge, CancellationToken cancellationToken) in C:\Users\kidon\Desktop\planetarium_repo\NineChronicles.Headless\Lib9c\.Libplanet\Libplanet\Net\Swarm.cs:line 1672

Problem

When peers are no longer available on NineChronicles.Headless due to various reasons (i.e. Internet disconnection), System.InvalidOperationException: The operation is not allowed on non-connected sockets error keeps occurring and peers cannot be reconnected even when the issue is resolved.

How to Reproduce

Run NineChronicles.Headless in debug mode.
Query PeerChainState on http://localhost:23061/ui/playground and make sure peers are connected.
Disconnect the Internet.
Re-query PeerChainState on http://localhost:23061/ui/playground and check that peers are not connected and find System.InvalidOperationException: The operation is not allowed on non-connected sockets. on the debug log.
Reconnect the Internet.

Expected Result

Once the Internet is reconnected, re-querying PeerChainState on http://localhost:23061/ui/playground should restore reconnected peers and System.InvalidOperationException: The operation is not allowed on non-connected sockets. should no longer appear the debug log.

Actual Result

However, the PeerChainState query on http://localhost:23061/ui/playground shows that peers are still unavailable even after reconnecting to the Internet and System.InvalidOperationException: The operation is not allowed on non-connected sockets. continues to appear on the debug log.

Incomplete fix

Wrapping https://github.com/planetarium/libplanet/blob/main/Libplanet.Stun/Stun/TurnClient.cs#L160-L177 with a try-catch block to ignore the InvalidOperationException does enable PeerChainState to be restored after reconnecting to the Internet.

Example

    public async Task CreatePermissionAsync(
        IPEndPoint peerAddress,
        CancellationToken cancellationToken = default(CancellationToken))
    {
        try
        {
            NetworkStream stream = _control.GetStream();
            var request = new CreatePermissionRequest(peerAddress);
            await SendMessageAsync(stream, request, cancellationToken);
            StunMessage response = await ReceiveMessageAsync(
                request.TransactionId,
                cancellationToken);

            if (response is CreatePermissionErrorResponse)
            {
                throw new TurnClientException(
                    "CreatePermission failed.",
                    response);
            }
        }
        catch (InvalidOperationException ex)
        {
            Log.Debug("Error Message: {0}", ex.Message);
        }
    }

However, the System.InvalidOperationException: The operation is not allowed on non-connected sockets. continues to appear on the debug log despite the restored peer connection.
NOTE! ) Even after the PeerChainState is restored, the headless client is unable to receive new blocks from its peers.
I think that the non-connected sockets should be cleaned up once the peer connection is restored but I'm not entirely sure how and where in the Libplanet code to implement this.

planetarium / libplanet