multipath-tcp / mptcp_net-next

Development version of the Upstream MultiPath TCP Linux kernel 🐧
https://mptcp.dev
Other
290 stars 41 forks source link

[6.1] selftests: `mptcp_connect.sh` failures with `copyfd_io_poll: poll timed out` errors #483

Open matttbe opened 7 months ago

matttbe commented 7 months ago

nkambo from LKFT reported that mptcp_connect.sh subtests are, e.g.

# INFO: with peek mode: saveWithPeek
# ns1 MPTCP -> ns1 (10.0.1.1:10042      ) MPTCP (duration  1024ms) [ OK ]
# ns1 MPTCP -> ns1 (10.0.1.1:10043      ) TCP   (duration   920ms) [ OK ]
# ns1 TCP   -> ns1 (10.0.1.1:10044      ) MPTCP (duration   946ms) [ OK ]
# ns1 MPTCP -> ns1 (dead:beef:1::1:10045) MPTCP copyfd_io_poll: poll timed out (events: POLLIN 1, POLLOUT 0)
# copyfd_io_poll: poll timed out (events: POLLIN 1, POLLOUT 4)
# (duration 30715ms) [ FAIL ] client exit code 2, server 0
# 
# netns ns1-660af233-2t1WrG socket stat for 10045:
# Failed to find cgroup2 mount
# Failed to find cgroup2 mount
# Failed to find cgroup2 mount
# Netid State      Recv-Q Send-Q    Local Address:Port      Peer Address:Port Process                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            
# tcp   ESTAB      0      0      [dead:beef:1::1]:10045 [dead:beef:1::1]:46080 ino:0 sk:1 cgroup:unreachable:100000270 <->
#    ts sack cubic wscale:7,7 rto:210 rtt:2.796/3.618 ato:40 mss:32768 pmtu:65536 rcvmss:32740 advmss:65464 cwnd:10 bytes_sent:3551908 bytes_acked:3551908 bytes_received:3453160 segs_out:269 segs_in:276 data_segs_out:198 data_segs_in:205 send 937567954bps lastsnd:32070 lastrcv:32070 lastack:32070 pacing_rate 1874716840bps delivery_rate 3084047056bps delivered:199 busy:370ms rcv_rtt:1.584 rcv_space:65464 rcv_ssthresh:65464 minrtt:0.106 tcp-ulp-mptcp flags:Mec token:0000(id:0)/d5bd3a39(id:0) seq:f37176dc5294319 sfseq:34b069 ssnoff:674db49d maplen:80
# mptcp FIN-WAIT-1 0      0      [dead:beef:1::1]:10045 [dead:beef:1::1]:46080 timer:(keepalive,56sec,0) ino:0 sk:2 cgroup:unreachable:100000270 ---
#    subflows_max:2 remote_key token:d5bd3a39 write_seq:9977e40e7e7017e9 snd_una:9977e40e7e556a8c rcv_nxt:f37176dc5294399                                                                                                                                                                                                                                                                                                                                                                                                                      
# TcpActiveOpens                  1                  0.0
# TcpPassiveOpens                 1                  0.0
# TcpInSegs                       546                0.0
# TcpOutSegs                      546                0.0
# TcpExtTCPPureAcks               141                0.0
# TcpExtTCPFromZeroWindowAdv      32                 0.0
# TcpExtTCPToZeroWindowAdv        34                 0.0
# TcpExtTCPWantZeroWindowAdv      118                0.0
# TcpExtTCPOrigDataSent           403                0.0
# TcpExtTCPDelivered              404                0.0
# MPTcpExtMPCapableSYNRX          1                  0.0
# MPTcpExtMPCapableSYNTX          1                  0.0
# MPTcpExtMPCapableSYNACKRX       1                  0.0
# MPTcpExtMPCapableACKRX          1                  0.0
# 
# netns ns1-660af233-2t1WrG socket stat for 10045:
# Failed to find cgroup2 mount
# Failed to find cgroup2 mount
# Failed to find cgroup2 mount
# Netid State      Recv-Q Send-Q    Local Address:Port      Peer Address:Port Process                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  
# tcp   ESTAB      0      0      [dead:beef:1::1]:46080 [dead:beef:1::1]:10045 ino:0 sk:1001 cgroup:unreachable:100000270 <->
#    ts sack cubic wscale:7,7 rto:210 rtt:1.809/2.461 ato:40 mss:32768 pmtu:65536 rcvmss:32740 advmss:65464 cwnd:10 bytes_sent:3453160 bytes_acked:3453161 bytes_received:3551908 segs_out:276 segs_in:270 data_segs_out:205 data_segs_in:198 send 1449110006bps lastsnd:34880 lastrcv:34880 lastack:34880 pacing_rate 2897419176bps delivery_rate 903944824bps delivered:206 busy:410ms rcv_rtt:0.29 rcv_space:65476 rcv_ssthresh:65476 minrtt:0.129 tcp-ulp-mptcp flags:Mmec token:0000(id:0)/df3dad91(id:0) seq:9977e40e7e554bf0 sfseq:361409 ssnoff:ea61095a maplen:1e9c
# mptcp FIN-WAIT-1 0      0      [dead:beef:1::1]:46080 [dead:beef:1::1]:10045 timer:(keepalive,55sec,0) ino:0 sk:1002 cgroup:unreachable:100000270 ---
#    subflows_max:2 remote_key token:df3dad91 write_seq:f37176dc53656ce snd_una:f37176dc5294399 rcv_nxt:9977e40e7e556a8c                                                                                                                                                                                                                                                                                                                                                                                                                          
# 
# INFO: with peek mode: saveAfterPeek
# ns1 MPTCP -> ns1 (10.0.1.1:10046      ) MPTCP (duration  1180ms) [ OK ]
# ns1 MPTCP -> ns1 (10.0.1.1:10047      ) TCP   (duration  1086ms) [ OK ]
# ns1 TCP   -> ns1 (10.0.1.1:10048      ) MPTCP (duration   974ms) [ OK ]
# ns1 MPTCP -> ns1 (dead:beef:1::1:10049) MPTCP copyfd_io_poll: poll timed out (events: POLLIN 1, POLLOUT 4)
# copyfd_io_poll: poll timed out (events: POLLIN 1, POLLOUT 4)
# (duration 30398ms) [ FAIL ] client exit code 2, server 0
# 
# netns ns1-660af233-2t1WrG socket stat for 10049:
# Failed to find cgroup2 mount
# Failed to find cgroup2 mount
# Failed to find cgroup2 mount
# Netid State      Recv-Q Send-Q    Local Address:Port      Peer Address:Port Process                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
# tcp   ESTAB      0      0      [dead:beef:1::1]:10049 [dead:beef:1::1]:38402 ino:0 sk:3 cgroup:unreachable:100000270 <->
#    ts sack cubic wscale:7,7 rto:210 rtt:2.498/2.553 ato:40 mss:32768 pmtu:65536 rcvmss:32740 advmss:65464 cwnd:10 bytes_sent:565376 bytes_acked:565376 bytes_received:681416 segs_out:57 segs_in:57 data_segs_out:44 data_segs_in:45 send 1049415532bps lastsnd:30420 lastrcv:30420 lastack:30420 pacing_rate 2098411040bps delivery_rate 143326400bps delivered:45 busy:80ms rcv_rtt:1.829 rcv_space:65464 rcv_ssthresh:65464 minrtt:0.159 tcp-ulp-mptcp flags:Mec token:0000(id:0)/abb95786(id:0) seq:ca9a312f220f7188 sfseq:a39ad ssnoff:51dfee0 maplen:2c1c
# mptcp FIN-WAIT-1 0      0      [dead:beef:1::1]:10049 [dead:beef:1::1]:38402 timer:(keepalive,59sec,0) ino:0 sk:4 cgroup:unreachable:100000270 ---
#    subflows_max:2 remote_key token:abb95786 write_seq:ffd6fdaf3de59bf9 snd_una:ffd6fdaf3dcadc78 rcv_nxt:ca9a312f220f9da4                                                                                                                                                                                                                                                                                                                                                                                                             
# TcpActiveOpens                  1                  0.0
# TcpPassiveOpens                 1                  0.0
# TcpInSegs                       115                0.0
# TcpOutSegs                      115                0.0
# TcpExtTCPPureAcks               24                 0.0
# TcpExtTCPFromZeroWindowAdv      5                  0.0
# TcpExtTCPToZeroWindowAdv        7                  0.0
# TcpExtTCPWantZeroWindowAdv      12                 0.0
# TcpExtTCPOrigDataSent           89                 0.0
# TcpExtTCPDelivered              90                 0.0
# MPTcpExtMPCapableSYNRX          1                  0.0
# MPTcpExtMPCapableSYNTX          1                  0.0
# MPTcpExtMPCapableSYNACKRX       1                  0.0
# MPTcpExtMPCapableACKRX          1                  0.0
# 
# netns ns1-660af233-2t1WrG socket stat for 10049:
# Failed to find cgroup2 mount
# Failed to find cgroup2 mount
# Failed to find cgroup2 mount
# Netid State      Recv-Q Send-Q    Local Address:Port      Peer Address:Port Process                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
# tcp   ESTAB      0      0      [dead:beef:1::1]:38402 [dead:beef:1::1]:10049 ino:0 sk:5 cgroup:unreachable:100000270 <->
#    ts sack cubic wscale:7,7 rto:210 rtt:2.076/2.936 ato:40 mss:32768 pmtu:65536 rcvmss:32740 advmss:65464 cwnd:10 bytes_sent:681416 bytes_acked:681417 bytes_received:565376 segs_out:57 segs_in:58 data_segs_out:45 data_segs_in:44 send 1262736031bps lastsnd:30760 lastrcv:30760 lastack:30760 pacing_rate 2524560008bps delivery_rate 2072284584bps delivered:46 busy:60ms rcv_rtt:2.991 rcv_space:65476 rcv_ssthresh:65476 minrtt:0.121 tcp-ulp-mptcp flags:Mmec token:0000(id:0)/c395bc7e(id:0) seq:ffd6fdaf3dcadbf8 sfseq:8a001 ssnoff:f6b36c5e maplen:80
# mptcp FIN-WAIT-1 0      0      [dead:beef:1::1]:38402 [dead:beef:1::1]:10049 timer:(keepalive,59sec,0) ino:0 sk:6 cgroup:unreachable:100000270 ---
#    subflows_max:2 remote_key token:c395bc7e write_seq:ca9a312f2229f7dd snd_una:ca9a312f220f9da4 rcv_nxt:ffd6fdaf3dcadc78                                                                                                                                                                                                                                                                                                                                                                                                              
# 
# FAIL: Tests with peek mode have failed
# INFO: TFO not supported by the kernel: SKIP
# INFO: test tproxy ipv4
# ns1 MPTCP -> ns2 (10.0.3.1:20000      ) MPTCP (duration  1195ms) [ OK ]
# PASS: tproxy ipv4

From: https://qa-reports.linaro.org/lkft/linux-stable-rc-linux-6.1.y/build/v6.1.83-273-gaa2042702765/testrun/23261389/suite/kselftest-net-mptcp/test/net_mptcp_mptcp_connect_sh_-_mptcp_connect_peek_mode_saveAfterPeek_ns1_MPTCP__ns1_dead_beef_1_1_10049_MPTCP/log

Or

# PASS: tproxy ipv6
# INFO: disconnect
# ns1 MPTCP -> ns1 (10.0.1.1:20001      ) MPTCP (duration   492ms) [ OK ]
# ns1 MPTCP -> ns1 (10.0.1.1:20002      ) TCP   (duration   453ms) [ OK ]
# ns1 TCP   -> ns1 (10.0.1.1:20003      ) MPTCP (duration   440ms) [ OK ]
# ns1 MPTCP -> ns1 (dead:beef:1::1:20004) MPTCP copyfd_io_poll: poll timed out (events: POLLIN 1, POLLOUT 0)
# copyfd_io_poll: poll timed out (events: POLLIN 1, POLLOUT 0)
# (duration 30472ms) [ FAIL ] client exit code 2, server 0
# 
# netns ns1-660aef6d-m4rGoz socket stat for 20004:
# Failed to find cgroup2 mount
# Failed to find cgroup2 mount
# Failed to find cgroup2 mount
# Netid State      Recv-Q Send-Q    Local Address:Port      Peer Address:Port Process                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
# tcp   ESTAB      0      0      [dead:beef:1::1]:20004 [dead:beef:1::1]:50964 ino:0 sk:1 cgroup:unreachable:34b <->
#    ts sack cubic wscale:7,7 rto:202 rtt:1.263/1.282 ato:40 mss:32768 pmtu:65536 rcvmss:32740 advmss:65464 cwnd:10 bytes_sent:375908 bytes_acked:375908 bytes_received:538440 segs_out:41 segs_in:40 data_segs_out:29 data_segs_in:30 send 2075566112bps lastsnd:30637 lastrcv:30637 lastack:30637 pacing_rate 4149079032bps delivery_rate 963764704bps delivered:30 busy:31ms rcv_rtt:1.706 rcv_space:65464 rcv_ssthresh:65464 minrtt:0.092 tcp-ulp-mptcp flags:Mec token:0000(id:0)/954600f3(id:0) seq:27efec917ad6f3df sfseq:82cad ssnoff:6c20c270 maplen:a9c
# mptcp FIN-WAIT-1 0      0      [dead:beef:1::1]:20004 [dead:beef:1::1]:50964 timer:(keepalive,59sec,0) ino:0 sk:1001 cgroup:unreachable:34b ---
#    subflows_max:2 remote_key token:954600f3 write_seq:3345f323ae44667a snd_una:3345f323ae3be6c1 rcv_nxt:27efec917ad6fe7b                                                                                                                                                                                                                                                                                                                                                                                                          
# TcpActiveOpens                  3                  0.0
# TcpPassiveOpens                 3                  0.0
# TcpEstabResets                  2                  0.0
# TcpInSegs                       488                0.0
# TcpOutSegs                      488                0.0
# TcpOutRsts                      2                  0.0
# TcpExtTCPPureAcks               94                 0.0
# TcpExtTCPFromZeroWindowAdv      17                 0.0
# TcpExtTCPToZeroWindowAdv        19                 0.0
# TcpExtTCPWantZeroWindowAdv      50                 0.0
# TcpExtTCPOrigDataSent           386                0.0
# TcpExtTCPDelivered              387                0.0
# MPTcpExtMPCapableSYNRX          3                  0.0
# MPTcpExtMPCapableSYNTX          3                  0.0
# MPTcpExtMPCapableSYNACKRX       3                  0.0
# MPTcpExtMPCapableACKRX          3                  0.0
# MPTcpExtMPFastcloseTx           2                  0.0
# MPTcpExtMPFastcloseRx           2                  0.0
# MPTcpExtMPRstTx                 2                  0.0
# MPTcpExtMPRstRx                 2                  0.0
# MPTcpExtRcvWndShared            2                  0.0
# 
# netns ns1-660aef6d-m4rGoz socket stat for 20004:
# Failed to find cgroup2 mount
# Failed to find cgroup2 mount
# Failed to find cgroup2 mount
# Netid State      Recv-Q Send-Q    Local Address:Port      Peer Address:Port Process                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  
# tcp   ESTAB      0      0      [dead:beef:1::1]:50964 [dead:beef:1::1]:20004 ino:0 sk:1002 cgroup:unreachable:34b <->
#    ts sack cubic wscale:7,7 rto:201 rtt:0.97/1.273 ato:40 mss:32768 pmtu:65536 rcvmss:32740 advmss:65464 cwnd:10 bytes_sent:538440 bytes_acked:538441 bytes_received:375908 segs_out:40 segs_in:42 data_segs_out:30 data_segs_in:29 send 2702515464bps lastsnd:31170 lastrcv:31170 lastack:31170 pacing_rate 5403638232bps delivery_rate 2394009128bps delivered:31 busy:138ms rcv_rtt:1.591 rcv_space:65476 rcv_ssthresh:65476 minrtt:0.099 tcp-ulp-mptcp flags:Mmec token:0000(id:0)/72bafc5e(id:0) seq:3345f323ae3be641 sfseq:5bbe5 ssnoff:b1fe3af4 maplen:80
# mptcp FIN-WAIT-1 0      0      [dead:beef:1::1]:50964 [dead:beef:1::1]:20004 timer:(keepalive,58sec,0) ino:0 sk:1003 cgroup:unreachable:34b ---
#    subflows_max:2 remote_key token:72bafc5e write_seq:27efec917add0350 snd_una:27efec917ad6fe7b rcv_nxt:3345f323ae3be6c1                                                                                                                                                                                                                                                                                                                                                                                                              
# 
# FAIL: Tests of the full disconnection have failed

From: https://qa-reports.linaro.org/lkft/linux-stable-rc-linux-6.1.y/build/v6.1.83-273-gaa2042702765/testrun/23259658/suite/kselftest-net-mptcp/test/net_mptcp_mptcp_connect_sh_-_mptcp_connect_full_disconnect_ns1_MPTCP__ns1_dead_beef_1_1_20004_MPTCP/log

Looking at the history, the mptcp_connect.sh test seems quite unstable: https://qa-reports.linaro.org/lkft/linux-stable-rc-linux-6.1.y/build/v6.1.83-273-gaa2042702765/testrun/23259658/suite/kselftest-net-mptcp/test/net_mptcp_mptcp_connect_sh/history/

matttbe commented 7 months ago

I don't know if this one is linked to it, but one MPTCP Join is also "regularly" failing:

# 112 userspace pm add & remove address
#       syn                                 [ ok ]
#       synack                              [ ok ]
#       ack                                 [ ok ]
#       add                                 [ ok ]
#       echo                                [ ok ]
#       mptcp_info subflows=2:2             [ ok ]
#       subflows_total 3:3                  [ ok ]
#       mptcp_info add_addr_signal=2:2      [ ok ]
# netlink error -3 (No such process)
# ./pm_nl_ctl: bailing out due to netlink error[s]
#       rm                                  [ ok ]
#       rmsf                                [fail] got 1 RM_SUBFLOW[s] expected 2
# Server ns stats
# TcpPassiveOpens                 3                  0.0
# TcpEstabResets                  1                  0.0
# TcpInSegs                       249                0.0
# TcpOutSegs                      200                0.0
# TcpExtDelayedACKs               39                 0.0
# TcpExtTCPPureAcks               116                0.0
# TcpExtTCPOrigDataSent           130                0.0
# TcpExtTCPDelivered              129                0.0
# MPTcpExtMPCapableSYNRX          1                  0.0
# MPTcpExtMPCapableACKRX          1                  0.0
# MPTcpExtMPJoinSynRx             2                  0.0
# MPTcpExtMPJoinAckRx             2                  0.0
# MPTcpExtEchoAdd                 2                  0.0
# MPTcpExtRmSubflow               1                  0.0
# Client ns stats
# TcpActiveOpens                  3                  0.0
# TcpInSegs                       206                0.0
# TcpOutSegs                      253                0.0
# TcpOutRsts                      1                  0.0
# TcpExtTW                        2                  0.0
# TcpExtDelayedACKs               49                 0.0
# TcpExtTCPPureAcks               69                 0.0
# TcpExtTCPOrigDataSent           132                0.0
# TcpExtTCPDelivered              135                0.0
# MPTcpExtMPCapableSYNTX          1                  0.0
# MPTcpExtMPCapableSYNACKRX       1                  0.0
# MPTcpExtMPJoinSynAckRx          2                  0.0
# MPTcpExtDuplicateData           1                  0.0
# MPTcpExtAddAddr                 2                  0.0
# MPTcpExtRmAddr                  2                  0.0
# MPTcpExtRcvWndShared            2                  0.0
#       Info: invert
#       mptcp_info subflows=0:0             [ ok ]
#       subflows_total 1:1                  [ ok ]

https://qa-reports.linaro.org/lkft/linux-stable-rc-linux-6.1.y/build/v6.1.83-273-gaa2042702765/testrun/23260836/suite/kselftest-net-mptcp/test/net_mptcp_mptcp_join_sh_-_mptcp_join_userspace_pm_add_remove_address/log

The counter TcpEstabResets is also incremented here in case of error, maybe a link?

The pm_nl_ctl failure is linked to this:

      mptcp_info add_addr_signal=2:2      [ ok ]                                                                                    
+ ip netns exec ns1-660c230c-HNYZEv ./pm_nl_ctl rem token 420034353 id 10                                                           
+ set +x                                                                                                                            
+ ip netns exec ns1-660c230c-HNYZEv ./pm_nl_ctl dsf lip ::ffff:10.0.2.1 lport 10111 rip ::ffff:10.0.2.2 rport 47945 token 420034353 
+ set +x                                                                                                                            
+ ip netns exec ns1-660c230c-HNYZEv ./pm_nl_ctl rem token 420034353 id 20                                                           
+ set +x                                                                                                                            
+ ip netns exec ns1-660c230c-HNYZEv ./pm_nl_ctl dsf lip 10.0.3.1 lport 10111 rip ::ffff:10.0.3.2 rport 56869 token 420034353        
netlink error -3 (No such process)                                                                                                  
./pm_nl_ctl: bailing out due to netlink error[s]                                                                                    
+ set +x                                                                                                                            
      rm                                  [ ok ]                                                                                    
      rmsf                                [fail] got 1 RM_SUBFLOW[s] expected 2

So only the 4th command on the server side failed, because it didn't find the subflow for this connection (-3 error). Probably because the subflow has been reset?

If we look at the events, the subflow has been removed (MPTCP_EVENT_SUB_CLOSED = 11):

+ ip netns exec ns1-660c34a1-ebiad5 ./pm_nl_ctl dsf lip ::ffff:10.0.2.1 lport 10111 rip ::ffff:10.0.2.2 rport 50643 token 226034464
type:11,token:226034464,family:10,saddr6:::ffff:10.0.2.1,daddr6:::ffff:10.0.2.2,sport:10111,dport:50643,loc_id:10,rem_id:1,backup:0
(...)
+ ip netns exec ns1-660c34a1-ebiad5 ./pm_nl_ctl dsf lip 10.0.3.1 lport 10111 rip ::ffff:10.0.3.2 rport 36365 token 226034464
netlink error -3 (No such process)
./pm_nl_ctl: bailing out due to netlink error[s]
type:11,token:226034464,family:10,saddr6:::ffff:10.0.3.1,daddr6:::ffff:10.0.3.2,sport:10111,dport:36365,loc_id:20,rem_id:2,backup:0
matttbe commented 7 months ago

Here are two traces when having the last issue with mptcp_join:

It is interesting to look at the 3rd stream (tcp.stream eq 2):

# tcpdump -nr mp_join-112-ns1-ok-660c371d-EHjLjO.pcap tcp and port 57515
reading from file mp_join-112-ns1-ok-660c371d-EHjLjO.pcap, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 65535
Warning: interface names might be incorrect
18:49:36.661287 docker0 In  IP 10.0.3.2.57515 > 10.0.3.1.10111: Flags [S], seq 2867335566, win 64256, options [mss 1460,sackOK,TS val 740440611 ecr 0,nop,wscale 7,mptcp 12 join id 2 token 0x4483f5d0 nonce 0x54a0aa8b], length 0
18:49:36.661298 docker0 Out IP 10.0.3.1.10111 > 10.0.3.2.57515: Flags [S.], seq 4277192773, ack 2867335567, win 65160, options [mss 1460,sackOK,TS val 1897770162 ecr 740440611,nop,wscale 7,mptcp 16 join id 20 hmac 0xfa7ce90421245e94 nonce 0xfdf38d4e], length 0
18:49:36.661313 docker0 In  IP 10.0.3.2.57515 > 10.0.3.1.10111: Flags [.], ack 1, win 502, options [nop,nop,TS val 740440611 ecr 1897770162,mptcp 24 join hmac 0xb601d40e5ec37a2f295fbfbc4989cd13d39a07fa], length 0
18:49:36.661331 docker0 Out IP 10.0.3.1.10111 > 10.0.3.2.57515: Flags [.], ack 1, win 510, options [nop,nop,TS val 1897770162 ecr 740440611,mptcp 12 dss ack 13289830844751537108], length 0
18:49:39.333176 docker0 In  IP 10.0.3.2.57515 > 10.0.3.1.10111: Flags [.], ack 1, win 502, options [nop,nop,TS val 740440611 ecr 1897770162,mptcp 12 dss ack 3640936710867625463], length 0
18:49:39.333197 docker0 In  IP 10.0.3.2.57515 > 10.0.3.1.10111: Flags [F.], seq 1, ack 1, win 502, options [nop,nop,TS val 740443283 ecr 1897770162,mptcp 12 dss ack 3640936710867625463], length 0
18:49:39.334593 docker0 Out IP 10.0.3.1.10111 > 10.0.3.2.57515: Flags [.], ack 2, win 510, options [nop,nop,TS val 1897772836 ecr 740443283,mptcp 12 dss ack 13289830844751537173], length 0
18:49:40.666604 docker0 Out IP 10.0.3.1.10111 > 10.0.3.2.57515: Flags [.], ack 2, win 510, options [nop,nop,TS val 1897772836 ecr 740443283,mptcp 12 dss ack 13289830844751537208], length 0
18:49:40.666631 docker0 Out IP 10.0.3.1.10111 > 10.0.3.2.57515: Flags [F.], seq 1, ack 2, win 510, options [nop,nop,TS val 1897774168 ecr 740443283,mptcp 12 dss ack 13289830844751537208], length 0
18:49:40.666640 docker0 In  IP 10.0.3.2.57515 > 10.0.3.1.10111: Flags [.], ack 2, win 502, options [nop,nop,TS val 740444617 ecr 1897774168], length 0
# tcpdump -nr mp_join-112-ns1-nok-660c372c-KD5wmI.pcap tcp and port 52717 
reading from file mp_join-112-ns1-nok-660c372c-KD5wmI.pcap, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 65535
Warning: interface names might be incorrect
18:49:50.933196 docker0 In  IP 10.0.3.2.52717 > 10.0.3.1.10111: Flags [S], seq 1030980353, win 64256, options [mss 1460,sackOK,TS val 740454883 ecr 0,nop,wscale 7,mptcp 12 join id 2 token 0x3d144ed6 nonce 0x7743f7a4], length 0
18:49:50.933219 docker0 Out IP 10.0.3.1.10111 > 10.0.3.2.52717: Flags [S.], seq 2375628578, ack 1030980354, win 65160, options [mss 1460,sackOK,TS val 1897784434 ecr 740454883,nop,wscale 7,mptcp 16 join id 20 hmac 0x7fe4b626c5cd422c nonce 0xca2ce698], length 0
18:49:50.933245 docker0 In  IP 10.0.3.2.52717 > 10.0.3.1.10111: Flags [.], ack 1, win 502, options [nop,nop,TS val 740454883 ecr 1897784434,mptcp 24 join hmac 0xc7a0c08aeed8af905201939a58a0b39356907dc2], length 0
18:49:50.933272 docker0 Out IP 10.0.3.1.10111 > 10.0.3.2.52717: Flags [.], ack 1, win 510, options [nop,nop,TS val 1897784434 ecr 740454883,mptcp 12 dss ack 3014908832125989087], length 0
18:49:53.548318 docker0 Out IP 10.0.3.1.10111 > 10.0.3.2.52717: Flags [P.], seq 1:6, ack 1, win 510, options [nop,nop,TS val 1897787049 ecr 740454883,mptcp 26 dss ack 3014908832125989152 seq 7370911507231076069 subseq 1 len 5,nop,nop], length 5
18:49:53.548354 docker0 In  IP 10.0.3.2.52717 > 10.0.3.1.10111: Flags [.], ack 6, win 502, options [nop,nop,TS val 740457498 ecr 1897787049,mptcp 12 dss ack 7370911507231076074], length 0
18:49:53.617220 docker0 In  IP 10.0.3.2.52717 > 10.0.3.1.10111: Flags [.], ack 6, win 502, options [nop,nop,TS val 740457498 ecr 1897787049,mptcp 12 dss ack 7370911507231076074], length 0
18:49:53.617238 docker0 In  IP 10.0.3.2.52717 > 10.0.3.1.10111: Flags [F.], seq 1, ack 6, win 502, options [nop,nop,TS val 740457567 ecr 1897787049,mptcp 12 dss ack 7370911507231076074], length 0
18:49:53.617620 docker0 Out IP 10.0.3.1.10111 > 10.0.3.2.52717: Flags [.], ack 2, win 510, options [nop,nop,TS val 1897787119 ecr 740457567,mptcp 12 dss ack 3014908832125989152], length 0
18:49:53.718637 docker0 Out IP 10.0.3.1.10111 > 10.0.3.2.52717: Flags [P.], seq 6:11, ack 2, win 510, options [nop,nop,TS val 1897787220 ecr 740457567,mptcp 26 dss ack 3014908832125989152 seq 7370911507231076074 subseq 6 len 5,nop,nop], length 5
18:49:53.718673 docker0 In  IP 10.0.3.2.52717 > 10.0.3.1.10111: Flags [R], seq 1030980355, win 0, length 0

When there is no issue, there is no data, just the 3WHS and the closure. When there is an issue, one end sends a packet with data just after having received the FIN → consequence: RST then.