search

openvswitch / ovs-issues

Issue tracker repo for Open vSwitch
10 stars 3 forks source link

Conntrack netlink event delivery is not correct. #282

Open yuntanghsu opened 1 year ago

yuntanghsu commented 1 year ago

When using conntrack -E to listen events for iperf3 traffic. Only one tcp update event (SYN_SENT) will be delivered. The other conntrack state change messages are not advertised.

I expect I can see the tcp state changes (SYN_SENT/SYN_RECV/ESTABLISHEDFIN_WAIT….etc) when I have a tcp connection. However, when I use iperf3 (iperf3 -c 10.244.2.9 -t 1) to have a tcp connection, I have following results:

In conntrack -E, there is only one tcp state (SYN_SENT). TIME_WAIT/CLOSE/CLOSE_WAIT is supposed to appear in events when a connection ends, but I can’t see any of them when using iperf3.

[1685571135.590407]  [NEW] tcp   6 120 SYN_SENT src=10.244.1.6 dst=10.244.2.9 sport=49400 dport=5201 [UNREPLIED] src=10.244.2.9 dst=10.244.1.6 sport=5201 dport=49400 zone=65520
[1685571135.590491] [UPDATE] tcp   6 120 src=10.244.1.6 dst=10.244.2.9 sport=49400 dport=5201 [UNREPLIED] src=10.244.2.9 dst=10.244.1.6 sport=5201 dport=49400 mark=3 zone=65520
[1685571135.592658]  [NEW] tcp   6 120 SYN_SENT src=10.244.1.6 dst=10.244.2.9 sport=49402 dport=5201 [UNREPLIED] src=10.244.2.9 dst=10.244.1.6 sport=5201 dport=49402 zone=65520
[1685571135.592697] [UPDATE] tcp   6 120 src=10.244.1.6 dst=10.244.2.9 sport=49402 dport=5201 [UNREPLIED] src=10.244.2.9 dst=10.244.1.6 sport=5201 dport=49402 mark=3 zone=65520

In conntrack -L, I can see the tcp state is changed to TIME_WAIT, which is supposed to be in conntrack events, but it isn’t.

tcp   6 104 TIME_WAIT src=10.244.1.6 dst=10.244.2.9 sport=52730 dport=5201 packets=18 bytes=1375 src=10.244.2.9 dst=10.244.1.6 sport=5201 dport=52730 packets=18 bytes=1247 [ASSURED] mark=3 zone=65520 delta-time=17 use=1
tcp   6 104 TIME_WAIT src=10.244.1.6 dst=10.244.2.9 sport=52732 dport=5201 packets=39669 bytes=1448346109 src=10.244.2.9 dst=10.244.1.6 sport=5201 dport=52732 packets=24397 bytes=1269164 [ASSURED] mark=3 zone=65520 delta-time=17 use=1

If I use wget, I can see all the tcp states in events:

[1685572749.172775]     [NEW] tcp      6 120 SYN_SENT src=10.244.1.6 dst=142.250.191.68 sport=44606 dport=80 [UNREPLIED] src=142.250.191.68 dst=172.18.0.3 sport=80 dport=44606
[1685572749.212094]  [UPDATE] tcp      6 60 SYN_RECV src=10.244.1.6 dst=142.250.191.68 sport=44606 dport=80 src=142.250.191.68 dst=172.18.0.3 sport=80 dport=44606
[1685572749.212144]  [UPDATE] tcp      6 432000 ESTABLISHED src=10.244.1.6 dst=142.250.191.68 sport=44606 dport=80 src=142.250.191.68 dst=172.18.0.3 sport=80 dport=44606 [ASSURED]
[1685572749.350889]  [UPDATE] tcp      6 120 FIN_WAIT src=10.244.1.6 dst=142.250.191.68 sport=44606 dport=80 src=142.250.191.68 dst=172.18.0.3 sport=80 dport=44606 [ASSURED]
[1685572749.351111]  [UPDATE] tcp      6 3600 CLOSE_WAIT src=10.244.1.6 dst=142.250.191.68 sport=44606 dport=80 src=142.250.191.68 dst=172.18.0.3 sport=80 dport=44606 [ASSURED]
[1685572749.385305]  [UPDATE] tcp      6 30 LAST_ACK src=10.244.1.6 dst=142.250.191.68 sport=44606 dport=80 src=142.250.191.68 dst=172.18.0.3 sport=80 dport=44606 [ASSURED]
[1685572749.385362]  [UPDATE] tcp      6 120 TIME_WAIT src=10.244.1.6 dst=142.250.191.68 sport=44606 dport=80 src=142.250.191.68 dst=172.18.0.3 sport=80 dport=44606 [ASSURED]

Here is a commit that fixed a similar issue.

ashish-varma commented 1 year ago

The description mentions conntrack and iperf. Can you mention the OVS use case here?

yuntanghsu commented 1 year ago

65520 is the zone used by Antrea (in OVS flows). Only flows with zone 65520 miss some events here. I think maybe conntrack event is updated by OVS plays a role here?

chaudron commented 1 year ago

@vlrpl any idea on this?

chaudron commented 1 year ago

@yuntanghsu maybe you can post your question in ovs-dev or ovs-discuss as more people might see it.