Socket.Connection refused on linux and osx

[dv00d00]

Not sure if it is a bug but this code works perfectly fine on windows:

[Fact]
public static void UDP_MultipleSends()
{
    var socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
    socket.Connect(new IPEndPoint(IPAddress.Loopback, 12345));

    for (int i = 0; i < 100; i++)
    {
        socket.Send(Encoding.ASCII.GetBytes("hello world"));
    }
}

but fails on linux

Operating System Details
Distributor ID: Ubuntu
Description:    Ubuntu 14.04.5 LTS
Release:    14.04
Codename:   trusty

and OSX

instance: 6963e4a4-daa1-404d-b285-a2db097e200f travis-ci-macos10.12-xcode8.3-1507738863 (via amqp)

dotnet version: 2.1.302

Error Message:
 System.Net.Sockets.SocketException : Connection refused

---

Related fact

SendTo behaves differently - see https://github.com/dotnet/corefx/issues/31206#issuecomment-406515367

[davidsh]

cc: @wfurt

[wfurt]

Do you have anything listening on that port @dv00d00 ? You may run tcpdump -ni lo port 12345 (or strace)

[wfurt]

It seems like the error is coming from OS:

strace -f -e network ~/dotnet-2.1.302/dotnet run
[pid 28081] connect(26, {sa_family=AF_INET, sin_port=htons(12345), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
Connect OK
[pid 28081] sendmsg(26, {msg_name(0)=NULL, msg_iov(1)=[{"hello world", 11}], msg_controllen=0, msg_flags=0}, 0) = 11
[pid 28081] sendmsg(26, {msg_name(0)=NULL, msg_iov(1)=[{"hello world", 11}], msg_controllen=0, msg_flags=MSG_DONTWAIT|MSG_WAITALL|MSG_CONFIRM|MSG_ERRQUEUE|MSG_NOSIGNAL|MSG_WAITFORONE|0x85500000}, 0) = -1 ECONNREFUSED (Connection refused)

When sending data locally, kernel can probably detect that port is closed and the sendmsg() call fails. That does not looks like problem with the runtime, more difference in how OS handle I/O.

[dv00d00]

On the other hand

[Fact]
public static void UDP_MultipleSends()
{
    var socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
    var ep = new IPEndPoint(IPAddress.Loopback, 12345);

    for (int i = 0; i < 100; i++)
    {
        socket.SendTo(Encoding.ASCII.GetBytes("hello world"), ep);
    }
}

works without throwing exceptions. I don't believe anything is on that specific port, this was happening on a travis CI machine, but the same happened on my mac.

[wfurt]

I did more testing with C and C# as well as I looked at Linux kernel code. This seems to be way how Unix works. when sendto() is used, individual chunks of data are submitted independently and the call succeeds as long as there is space in socket buffer. Since UDP is unreliable, this is has nothing to do with actual delivery.

I did also packet capture for both calls. In both cases I see:

furt@Ubuntu:~/git/wfurt-corefx-serial/src/System.Diagnostics.Process/src$ sudo tcpdump -eni lo
[sudo] password for furt:
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes
14:48:23.512034 00:00:00:00:00:00 > 00:00:00:00:00:00, ethertype IPv4 (0x0800), length 53: 127.0.0.1.46992 > 127.0.0.1.12345: UDP, length 11
14:48:23.512043 00:00:00:00:00:00 > 00:00:00:00:00:00, ethertype IPv4 (0x0800), length 81: 127.0.0.1 > 127.0.0.1: ICMP 127.0.0.1 udp port 12345 unreachable, length 47

When sendmsg() is trying to send data following happens: first message goes out without error. When the ICMP error get's back it is remembered on "connection" (internal socket structure)
Subsequent sendmsg() calls fail.

Since this is OS behavior, I don't think it make sense to hide underlying error. It seems that raising exception and allowing caller to deal with it is better approach.

I'm proposing to close this unless somebody objects. cc: @karelz (note that linked PR does not change this behavior)

[Krummelz]

Hi all, please take a look at this same issue over on the NpgSql repo - https://github.com/npgsql/npgsql/issues/2198. I'm experiencing the same thing. And their code is using the same Socket class to make the connections. You can look at the relevant Connect method code here: https://github.com/npgsql/npgsql/blob/91d23f90ef00eadc7c07966833959a5b3f877127/src/Npgsql/NpgsqlConnector.cs#L642.

"System.Net.Sockets.SocketException (111): Connection refused" error, at Npgsql.NpgsqlConnector.Connect(NpgsqlTimeout timeout)

I've run tcpdump -ni lo port 5432, and I get the following:

root@ubuntu-app-vm:~# tcpdump -ni lo port 5432
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes
10:18:57.878227 IP 127.0.0.1.50184 > 127.0.0.1.5432: Flags [S], seq 4146525521, win 43690, options [mss 65495,sackOK,TS val 2214584442 ecr 0,nop,wscale 7], length 0
10:18:57.878242 IP 127.0.0.1.5432 > 127.0.0.1.50184: Flags [R.], seq 0, ack 4146525522, win 0, length 0

[wfurt]

From the dump it is clear that the port you trying to connect to is not listening and sends back RST. It is correct to throw.
Also note that this issue is about UDP.

[karelz]

Triage: We need to understand why Send and SendTo behave differently. We need either doc change, or product change, or test added.

[vindicatorr]

I'm only just now getting into this as well and it looks like SCM_RIGHTS is going to be key: Send or receive a set of open file descriptors from another process. The data portion contains an integer array of the file descriptors.(unix.7) https://stackoverflow.com/questions/28003921/sending-file-descriptor-by-linux-socket/

[tmds]

This issue is unrelated to SCM_RIGHTS.

On Linux, Socket.Send throws a SocketException if there is no peer, while on Windows it does not.

Windows WSASend documentation suggests it could do the same thing:

WSAECONNRESET	For a stream socket, the virtual circuit was reset by the remote side. The application should close the socket as it is no longer usable. For a UDP datagram socket, this error would indicate that a previous send operation resulted in an ICMP "Port Unreachable" message.

Maybe this error is only generated for non-localhost communication on Windows?

You can add an extension method if you always want to ignore the error, and avoid the cost of throwing (and catching) Exceptions for it.

    static class SocketExtensions
    {
        public static int SendUdp(this Socket socket, ReadOnlySpan<byte> buffer)
        {
            int rv = socket.Send(buffer, SocketFlags.None, out SocketError socketError);
            return socketError switch
            {
                SocketError.Success => rv,
                SocketError.ConnectionRefused => 0,
                _ => throw new SocketException((int)socketError),
            };
        }
    }