enable udp SO_REUSEADDR to work with tproxy

[cattyhouse]

strace coredns shows that:

• for tcp both SO_REUSEADDR and SO_REUSEPORT are enabled
• for udp only SO_REUSEPORT is enabled.

is this by design? this would block linux tproxy from working when:

1. coredns listen on *.53, this is default
2. other program trying to query e.g. 1.1.1.1:53 udp
3. 1.1.1.1 has been set a tproxy mark
4. tproxy program will need to bind 1.1.1.1 on udp 53 on interface lo by design.
5. but coredns is already listening on udp *53, tproxy program will report failed to dial tproxy udp | dial: socket bind: address already in use
6. access 1.1.1.1:53 tcp does not have such issue.

if i need UDP SO_REUSEADDR, what modification do i need to make to get it working?

strace coredns:

socket(AF_INET6, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 3
setsockopt(3, SOL_IPV6, IPV6_V6ONLY, [0], 4) = 0
openat(AT_FDCWD, "/proc/sys/net/core/somaxconn", O_RDONLY|O_CLOEXEC) = 7
fcntl(7, F_GETFL)                       = 0x20000 (flags O_RDONLY|O_LARGEFILE)
fcntl(7, F_SETFL, O_RDONLY|O_NONBLOCK|O_LARGEFILE) = 0
epoll_ctl(4, EPOLL_CTL_ADD, 7, {events=EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, data={u32=1507315464, u64=281472189058824}}) = 0
read(7, "10240\n", 65536)               = 6
read(7, "", 65530)                      = 0
epoll_ctl(4, EPOLL_CTL_DEL, 7, 0x40000ed4d0) = 0
close(7)                                = 0
setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
setsockopt(3, SOL_SOCKET, SO_REUSEPORT, [1], 4) = 0
bind(3, {sa_family=AF_INET6, sin6_port=htons(53), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "::", &sin6_addr), sin6_scope_id=0}, 28) = 0
listen(3, 10240)                        = 0
epoll_ctl(4, EPOLL_CTL_ADD, 3, {events=EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, data={u32=1507315464, u64=281472189058824}}) = 0
getsockname(3, {sa_family=AF_INET6, sin6_port=htons(53), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "::", &sin6_addr), sin6_scope_id=0}, [112 => 28]) = 0
socket(AF_INET6, SOCK_DGRAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 7
setsockopt(7, SOL_IPV6, IPV6_V6ONLY, [0], 4) = 0
setsockopt(7, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0
futex(0xbb08e0, FUTEX_WAKE_PRIVATE, 1)  = 1
futex(0xbb07f8, FUTEX_WAKE_PRIVATE, 1)  = 1
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=7734, si_uid=0} ---
rt_sigreturn({mask=[]})                 = 12253056
setsockopt(7, SOL_SOCKET, SO_REUSEPORT, [1], 4) = 0
bind(7, {sa_family=AF_INET6, sin6_port=htons(53), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "::", &sin6_addr), sin6_scope_id=0}, 28) = 0
epoll_ctl(4, EPOLL_CTL_ADD, 7, {events=EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, data={u32=1507315224, u64=281472189058584}}) = 0
getsockname(7, {sa_family=AF_INET6, sin6_port=htons(53), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "::", &sin6_addr), sin6_scope_id=0}, [112 => 28]) = 0

[cattyhouse]

fixed with this patch:

diff --git a/plugin/pkg/reuseport/listen_reuseport.go b/plugin/pkg/reuseport/listen_reuseport.go
index 71fac3e7..6b3ba95a 100644
--- a/plugin/pkg/reuseport/listen_reuseport.go
+++ b/plugin/pkg/reuseport/listen_reuseport.go
@@ -17,6 +17,9 @@ func control(network, address string, c syscall.RawConn) error {
        if err := unix.SetsockoptInt(int(fd), unix.SOL_SOCKET, unix.SO_REUSEPORT, 1); err != nil {
            log.Warningf("Failed to set SO_REUSEPORT on socket: %s", err)
        }
+       if err := unix.SetsockoptInt(int(fd), unix.SOL_SOCKET, unix.SO_REUSEADDR, 1); err != nil {
+           log.Warningf("Failed to set SO_REUSEADDR on socket: %s", err)
+       }
    })
    return nil
 }

[cattyhouse]

similar functions in adguard/dnsproxy

https://github.com/AdguardTeam/dnsproxy/blob/4f1c527e4d60888d9509ad78872fef935270c861/internal/netutil/listenconfig_unix.go#L19

the strange part is, with coredns, the unix.SO_REUSEPORT also set SO_REUSEADDR for TCP, so if the above patch applied, SO_REUSEADDR gets set twice for TCP, but that's not a big problem

[cattyhouse]

@chrisohaver any comment prior 1.11.2 release?

[chrisohaver]

I'm not seeing this as a strong need. Perhaps I'm mistaken.

How long have you been using the patched build?

[cattyhouse]

I'm not seeing this as a strong need. Perhaps I'm mistaken.

How long have you been using the patched build?

Since May 3 , 2023