MPTCP can't be enabled with GoLang TCP client.

[tubzby]

Describe the bug I have a TCP client written in Golang and I find out that MPTCP can't be enabled by mptcpize.

Go: go version go1.18.3 linux/amd64 OS: Ubuntu 22.04.1 LTS

My code is just serval lines (make a tcp connection to 192.168.0.1:4323 and quit, server is not running btw):

package main

import (
    "net"
)

func main() {
    _, err := net.Dial("tcp", "192.168.0.1:4323")
    if err != nil {
        panic(err)
    }
}

Using mptcpsize to run, the output lacks mptcpize debug message which means it changed protocol to IPPROTO_MPTCP (Please ignore the error):

➜  gmptcp mptcpize run -d ./gmptcp
panic: dial tcp 192.168.0.1:4323: connect: connection refused

goroutine 1 [running]:
main.main()
    /home/XXX/src/gmptcp/main.go:10 +0x52

Using strace to find out the syscall:

➜  gmptcp strace -f ./gmptcp 2>&1 |grep socket
[pid 23111] socket(AF_INET, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP <unfinished ...>
[pid 23111] <... socket resumed>)       = 3

I have also verified TCP options via tcpdump, mptcp capable is not exist.

On the same machine, run iperf3 with mptcpsize, the output shows the protocol is changed(Please ignore the error):

➜  gmptcp mptcpize run -d iperf3 -c 192.168.0.1 -p 4323
mptcpwrap: changing socket protocol from 0x0 to 0x106 (IPPROTO_MPTCP) for family 0x2 type 0x1 fd 4
iperf3: error - unable to send control message: Bad file descriptor

Using strace to trace iperf3 socket syscall:

➜  gmptcp strace -f iperf3 -c 192.168.0.1 -p 4323 2>&1|grep socket
socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 4
➜  gmptcp

My guess is that the extra flags SOCK_CLOEXEC|SOCK_NONBLOCK added by GoLang std are causing this issue, am I right, how to avoid it?

To Reproduce Steps to reproduce the behavior:

1. go build
2. mptcpize run -d GOLANG_PROCESS

Expected behavior MPTCP can be enabled by mptcpize on Golang process.

Screenshots NA.

Desktop (please complete the following information):

• Linux Kernel: 5.15.0-47-generic
• Compiler:

Additional context Add any other context about the problem here.

[matttbe]

Hi @tubzby

I have a TCP client written in Golang and I find out that MPTCP can't be enabled by mptcpize. (...) My guess is that the extra flags SOCK_CLOEXEC|SOCK_NONBLOCK added by GoLang std are causing this issue, am I right, how to avoid it?

If I'm not mistaken, the issue is due to Go itself: mptcpize cannot work with Go applications.

mptcpize is defining LD_PRELOAD env var to change the behaviour of the linker to override some functions of the libc (here, the socket function). Go is not using a libc: they decided to re-implement syscall, etc. for various reasons. But because of that, this LD_PRELOAD technique here cannot work.

If you control your application, the best is to modify it to use MPTCP. You might have to modify net.Dial("tcp", "192.168.0.1:4323") to support a new argument ("mptcp"). I don't know if you can easily create a socket without using net.Dial() in Go.

[tubzby]

Thanks, @matttbe, I just found out how mptcpize was updating my systemd script :).

It's hard to modify the Go std lib and make the application hard to maintain, so I'm looking for other alternatives:

1. As described by redhat, maybe I can use stap instead.
2. Use a proxy like Nginx/HAProxy/Shadowsocket.

I haven't used stap before, can it hook my socket syscall?

[matttbe]

It's hard to modify the Go std lib and make the application hard to maintain

Is it something that could be reported to Go devs? We can help if a ticket is opened and if needed.

I haven't used stap before, can it hook my socket syscall?

Yes, I think stab should work in your case. I never had to use it with Go apps but from how it works (from what I heard), it should work.

[matttbe]

I think we can close this ticket:

• Go 1.21 supports MPTCP: https://github.com/golang/go/issues/56539
• An example of app in Go: https://github.com/golang/go/commit/383a4e78861218bd861632d6cdcac6c8af6a5470
• It is possible to GODEBUG=multipathtcp=1 to force an app to use MPTCP instead of TCP
• More details: https://github.com/golang/go/issues/56539 and https://github.com/golang/go/issues/59166