[Zeta] Data path integration

liangbin-pub commented 3 years ago

Track and communicate data path integration findings

liangbin-pub commented 3 years ago

Initial findings:

ZGC gateway entries' MAC addresses are not used for vxlan encapsulation, should be set using dl_dst as part of group table action
When I manually added those as static arp entry, the captured vxlan still shows problems:
- destination MAC is some strange value 00:04:00:01:00:06
- there are two extra bytes 0x0000 between outer source MAC and ethertype 0x0800 pcap: 0x0000: 0004 0001 0006 e8bd d101 77ec 0000 0800 0x0010: 4500 004e 24d8 4000 4011 6baf c0a8 145c 0x0020: c0a8 146b b57a 12b5 003a 0000 0800 0000 0x0030: 0003 7800 ffff ffff ffff 6cdd ee00 0000 0x0040: 0806 0001 0800 0604 0001 6cdd ee00 0000 0x0050: 7b00 0000 0000 0000 0000 7b00 0001

Minli may want to take a look before we come back tomorrow

Eric's note: two extra bytes 0x0000 between outer source MAC and ethertype 0x0800 was not seen on the compute host with the latest run, @zhangml don't need to investigate that yet. Please focus on the issue (1).

liangbin-pub commented 3 years ago

for reference: oot@fw001157:~# ovs-ofctl -O OpenFlow13 dump-groups br-tun 1 OFPST_GROUP_DESC reply (OF1.3) (xid=0x2): group_id=1,type=select,bucket=actions=set_field:192.168.20.113->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.111->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.110->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.109->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.114->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.108->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.107->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.106->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.115->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.101->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.102->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.100->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.103->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.112->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.104->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.105->tun_dst,output:"vxlan-generic"

er1cthe0ne commented 3 years ago

Non-zeta packet can be communicated with OVS fine and able to use tcpdump to capture the valid IPv4 tunnelled packet.

zeta packet is sent to zgc and the packet capture by @liangbin-pub show that it has extra "0000" before "0800" which make it an invalid packet.

I am seeing difference in how the Zeta openflow rule is constructed and understood by OVS:

int ACA_Zeta_Programming::_create_zeta_group_entry(zeta_config *zeta_cfg) { ...

// adding group table rule string cmd = "-O OpenFlow13 add-group br-tun group_id=" + to_string(zeta_cfg->group_id) + ",type=select"; unordered_set::iterator it; for (it = zeta_cfg->zeta_buckets.begin(); it != zeta_cfg->zeta_buckets.end(); it++) { cmd += ",bucket=\"set_field:" + *it + "->tun_dst,output:vxlan-generic\""; }

root@28abfb290c2e:/mnt/host/code/aca-zeta# ovs-ofctl dump-flows br-tun -O OpenFlow13 ... cookie=0x0, duration=2.493s, table=22, n_packets=0, n_bytes=0, reset_counts priority=50,dl_vlan=1 actions=pop_vlan,load:0x378->NXM_NX_TUN_ID[],group:1

ovs-ofctl dump-groups br-tun -O OpenFlow13 OFPST_GROUP_DESC reply (OF1.3) (xid=0x2): group_id=1,type=select,bucket=actions=set_field:192.168.20.113->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.111->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.110->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.109->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.114->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.108->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.107->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.106->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.115->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.101->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.102->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.100->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.103->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.112->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.104->tun_dst,output:"vxlan-generic",bucket=actions=set_field:192.168.20.105->tun_dst,output:"vxlan-generic"

for non-zeta ports:

int ACA_Vlan_Manager::create_l2_neighbor(string virtual_ip, string virtual_mac, string remote_host_ip, uint tunnel_id, ulong &culminative_time) { ... string cmd_string = "add-flow br-tun table=20,priority=50,dl_vlan=" + to_string(internal_vlan_id) + ",dl_dst:" + virtual_mac + ",\"actions=strip_vlan,load:" + to_string(tunnel_id) + "->NXM_NX_TUN_ID[],set_field:" + remote_host_ip + "->tun_dst,output:vxlan-generic\"";

ovs-ofctl dump-flows br-tun -O OpenFlow13 ... cookie=0x0, duration=732.671s, table=20, n_packets=0, n_bytes=0, reset_counts priority=50,dl_vlan=1,dl_dst=fa:16:3e:d7:f2:6c actions=pop_vlan,load:0x14->NXM_NX_TUN_ID[],load:0xac110002->NXM_NX_TUN_IPV4_DST[],output:"vxlan-generic"

zhangml commented 3 years ago

@er1cthe0ne Similar to CTSL::HashMap, is there a library file for hashSet?

er1cthe0ne commented 3 years ago

@zhangml - please note that the mac change (number 1 above) is highest priority and we need the change pretty quickly.

@er1cthe0ne Similar to CTSL::HashMap, is there a library file for hashSet?

This is second priority. You can still leverage CTSL::HashMap and use it similar to hashSet, please see:

namespace aca_vlan_manager { struct vpc_table_entry { uint vlan_id;

// list of ovs_ports names on this host in the same VPC to share the same internal vlan_id // CTSL::HashMap <key: ovs_port name, value: int (not used)> CTSL::HashMap<string, int > ovs_ports;

string auxGateway_id; };

liangbin-pub commented 3 years ago

Update as of 1/13/21: Ping respond was not captured at requester. Observation:

On bridge between ACAs and ZGC nodes, capture VxLAN packets we see ZGC is responding to ARP request correctly, forwarding ICMP ping request and respond correctly, see tcpdump below for complete exchange captured:


tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-tenant, link-type EN10MB (Ethernet), capture size 262144 bytes
14:50:02.553954 e8:bd:d1:01:77:ec > 82:83:c0:a8:14:6e, ethertype IPv4 (0x0800), length 148: 192.168.20.92.36828 > 192.168.20.110.4789: VXLAN, flags [I] (0x08), vni 888
6c:dd:ee:00:00:02 > 6c:dd:ee:00:00:01, ethertype IPv4 (0x0800), length 98: 123.0.0.2 > 123.0.0.1: ICMP echo request, id 21593, seq 1, length 64
0x0000:  8283 c0a8 146e e8bd d101 77ec 0800 4500  .....n....w...E.
0x0010:  0086 1d44 4000 4011 7308 c0a8 145c c0a8  ...D@.@.s....\..
0x0020:  146e 8fdc 12b5 0072 0000 0800 0000 0003  .n.....r........
0x0030:  7800 6cdd ee00 0001 6cdd ee00 0002 0800  x.l.....l.......
0x0040:  4500 0054 ca09 4000 4001 7a9c 7b00 0002  E..T..@.@.z.{...
0x0050:  7b00 0001 0800 7c80 5459 0001 1a79 ff5f  {.....|.TY...y._
0x0060:  0000 0000 4679 0800 0000 0000 1011 1213  ....Fy..........
0x0070:  1415 1617 1819 1a1b 1c1d 1e1f 2021 2223  .............!"#
0x0080:  2425 2627 2829 2a2b 2c2d 2e2f 3031 3233  $%&'()*+,-./0123
0x0090:  3435 3637                                4567
14:50:02.554047 e8:bd:d1:01:77:ec > e8:bd:d1:01:72:c8, ethertype IPv4 (0x0800), length 148: 192.168.20.92.36828 > 192.168.20.93.4789: VXLAN, flags [I] (0x08), vni 888
6c:dd:ee:00:00:02 > 6c:dd:ee:00:00:01, ethertype IPv4 (0x0800), length 98: 123.0.0.2 > 123.0.0.1: ICMP echo request, id 21593, seq 1, length 64
0x0000:  e8bd d101 72c8 e8bd d101 77ec 0800 4500  ....r.....w...E.
0x0010:  0086 1d44 4000 3f11 7419 c0a8 145c c0a8  ...D@.?.t....\..
0x0020:  145d 8fdc 12b5 0072 0000 0800 0000 0003  .].....r........
0x0030:  7800 6cdd ee00 0001 6cdd ee00 0002 0800  x.l.....l.......
0x0040:  4500 0054 ca09 4000 4001 7a9c 7b00 0002  E..T..@.@.z.{...
0x0050:  7b00 0001 0800 7c80 5459 0001 1a79 ff5f  {.....|.TY...y._
0x0060:  0000 0000 4679 0800 0000 0000 1011 1213  ....Fy..........
0x0070:  1415 1617 1819 1a1b 1c1d 1e1f 2021 2223  .............!"#
0x0080:  2425 2627 2829 2a2b 2c2d 2e2f 3031 3233  $%&'()*+,-./0123
0x0090:  3435 3637                                4567
14:50:02.554545 e8:bd:d1:01:72:c8 > 82:83:c0:a8:14:6e, ethertype IPv4 (0x0800), length 148: 192.168.20.93.46639 > 192.168.20.110.4789: VXLAN, flags [I] (0x08), vni 888
6c:dd:ee:00:00:01 > 6c:dd:ee:00:00:02, ethertype IPv4 (0x0800), length 98: 123.0.0.1 > 123.0.0.2: ICMP echo reply, id 21593, seq 1, length 64
0x0000:  8283 c0a8 146e e8bd d101 72c8 0800 4500  .....n....r...E.
0x0010:  0086 4607 4000 4011 4a44 c0a8 145d c0a8  ..F.@.@.JD...]..
0x0020:  146e b62f 12b5 0072 0000 0800 0000 0003  .n./...r........
0x0030:  7800 6cdd ee00 0002 6cdd ee00 0001 0800  x.l.....l.......
0x0040:  4500 0054 e775 0000 4001 9d30 7b00 0001  E..T.u..@..0{...
0x0050:  7b00 0002 0000 8480 5459 0001 1a79 ff5f  {.......TY...y._
0x0060:  0000 0000 4679 0800 0000 0000 1011 1213  ....Fy..........
0x0070:  1415 1617 1819 1a1b 1c1d 1e1f 2021 2223  .............!"#
0x0080:  2425 2627 2829 2a2b 2c2d 2e2f 3031 3233  $%&'()*+,-./0123
0x0090:  3435 3637                                4567
14:50:02.554598 e8:bd:d1:01:72:c8 > e8:bd:d1:01:77:ec, ethertype IPv4 (0x0800), length 148: 192.168.20.93.46639 > 192.168.20.92.4789: VXLAN, flags [I] (0x08), vni 888
6c:dd:ee:00:00:01 > 6c:dd:ee:00:00:02, ethertype IPv4 (0x0800), length 98: 123.0.0.1 > 123.0.0.2: ICMP echo reply, id 21593, seq 1, length 64
0x0000:  e8bd d101 77ec e8bd d101 72c8 0800 4500  ....w.....r...E.
0x0010:  0086 4607 4000 3f11 4b56 c0a8 145d c0a8  ..F.@.?.KV...]..
0x0020:  145c b62f 12b5 0072 0000 0800 0000 0003  .\./...r........
0x0030:  7800 6cdd ee00 0002 6cdd ee00 0001 0800  x.l.....l.......
0x0040:  4500 0054 e775 0000 4001 9d30 7b00 0001  E..T.u..@..0{...
0x0050:  7b00 0002 0000 8480 5459 0001 1a79 ff5f  {.......TY...y._
0x0060:  0000 0000 4679 0800 0000 0000 1011 1213  ....Fy..........
0x0070:  1415 1617 1819 1a1b 1c1d 1e1f 2021 2223  .............!"#
0x0080:  2425 2627 2829 2a2b 2c2d 2e2f 3031 3233  $%&'()*+,-./0123
0x0090:  3435 3637                                4567
14:50:07.589243 e8:bd:d1:01:72:c8 > 82:83:c0:a8:14:68, ethertype IPv4 (0x0800), length 92: 192.168.20.93.38768 > 192.168.20.104.4789: VXLAN, flags [I] (0x08), vni 888
6c:dd:ee:00:00:01 > 6c:dd:ee:00:00:02, ethertype ARP (0x0806), length 42: Request who-has 123.0.0.2 tell 123.0.0.1, length 28
0x0000:  8283 c0a8 1468 e8bd d101 72c8 0800 4500  .....h....r...E.
0x0010:  004e 45ff 4000 4011 4a8a c0a8 145d c0a8  .NE.@.@.J....]..
0x0020:  1468 9770 12b5 003a 0000 0800 0000 0003  .h.p...:........
0x0030:  7800 6cdd ee00 0002 6cdd ee00 0001 0806  x.l.....l.......
0x0040:  0001 0800 0604 0001 6cdd ee00 0001 7b00  ........l.....{.
0x0050:  0001 0000 0000 0000 7b00 0002            ........{...
14:50:07.589341 82:83:c0:a8:14:68 > e8:bd:d1:01:72:c8, ethertype IPv4 (0x0800), length 92: 192.168.20.104.38768 > 192.168.20.93.4789: VXLAN, flags [I] (0x08), vni 888
6c:dd:ee:00:00:02 > 6c:dd:ee:00:00:01, ethertype ARP (0x0806), length 42: Reply 123.0.0.2 is-at 6c:dd:ee:00:00:02, length 28
0x0000:  e8bd d101 72c8 8283 c0a8 1468 0800 4500  ....r......h..E.
0x0010:  004e 45ff 4000 3f11 4b8a c0a8 1468 c0a8  .NE.@.?.K....h..
0x0020:  145d 9770 12b5 003a 0000 0800 0000 0003  .].p...:........
0x0030:  7800 6cdd ee00 0001 6cdd ee00 0002 0806  x.l.....l.......
0x0040:  0001 0800 0604 0002 6cdd ee00 0002 7b00  ........l.....{.
0x0050:  0002 6cdd ee00 0001 7b00 0001            ..l.....{...
14:50:07.705044 e8:bd:d1:01:77:ec > 82:83:c0:a8:14:68, ethertype IPv4 (0x0800), length 92: 192.168.20.92.42216 > 192.168.20.104.4789: VXLAN, flags [I] (0x08), vni 888
6c:dd:ee:00:00:02 > 6c:dd:ee:00:00:01, ethertype ARP (0x0806), length 42: Request who-has 123.0.0.1 tell 123.0.0.2, length 28
0x0000:  8283 c0a8 1468 e8bd d101 77ec 0800 4500  .....h....w...E.
0x0010:  004e db3d 4000 4011 b54c c0a8 145c c0a8  .N.=@.@..L...\..
0x0020:  1468 a4e8 12b5 003a 0000 0800 0000 0003  .h.....:........
0x0030:  7800 6cdd ee00 0001 6cdd ee00 0002 0806  x.l.....l.......
0x0040:  0001 0800 0604 0001 6cdd ee00 0002 7b00  ........l.....{.
0x0050:  0002 0000 0000 0000 7b00 0001            ........{...
14:50:07.705123 82:83:c0:a8:14:68 > e8:bd:d1:01:77:ec, ethertype IPv4 (0x0800), length 92: 192.168.20.104.42216 > 192.168.20.92.4789: VXLAN, flags [I] (0x08), vni 888
6c:dd:ee:00:00:01 > 6c:dd:ee:00:00:02, ethertype ARP (0x0806), length 42: Reply 123.0.0.1 is-at 6c:dd:ee:00:00:01, length 28
0x0000:  e8bd d101 77ec 8283 c0a8 1468 0800 4500  ....w......h..E.
0x0010:  004e db3d 4000 3f11 b64c c0a8 1468 c0a8  .N.=@.?..L...h..
0x0020:  145c a4e8 12b5 003a 0000 0800 0000 0003  .\.....:........
0x0030:  7800 6cdd ee00 0002 6cdd ee00 0001 0806  x.l.....l.......
0x0040:  0001 0800 0604 0002 6cdd ee00 0001 7b00  ........l.....{.
0x0050:  0001 6cdd ee00 0002 7b00 0002            ..l.....{...


2.  On bridge between ACAs and ZGC nodes, captured OAM packets we generated and sent by ZGC:

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on ens4, link-type EN10MB (Ethernet), capture size 262144 bytes 23:04:12.291849 82:83:c0:a8:14:6e > e8:bd:d1:01:77:ec, ethertype IPv4 (0x0800), length 86: 192.168.20.110.0 > 192.168.20.92.8300: UDP, length 44 0x0000: e8bd d101 77ec 8283 c0a8 146e 0800 4500 ....w......n..E. 0x0010: 0048 d431 0000 3f11 fd58 c0a8 146e c0a8 .H.1..?..X...n.. 0x0020: 145c 0000 206c 0034 0000 0000 0000 7b00 ....l.4......{. 0x0030: 0002 7b00 0001 0000 0000 0100 0378 7b00 ..{..........x{. 0x0040: 0001 c0a8 145d 6cdd ee00 0001 e8bd d101 .....]l......... 0x0050: 72c8 001e 0000 r..... 23:04:12.291966 82:83:c0:a8:14:6e > e8:bd:d1:01:72:c8, ethertype IPv4 (0x0800), length 86: 192.168.20.110.0 > 192.168.20.93.8300: UDP, length 44 0x0000: e8bd d101 72c8 8283 c0a8 146e 0800 4500 ....r......n..E. 0x0010: 0048 d431 0000 3f11 fd57 c0a8 146e c0a8 .H.1..?..W...n.. 0x0020: 145d 0000 206c 0034 0000 0000 0000 7b00 .]...l.4......{. 0x0030: 0001 7b00 0002 0000 0000 0100 0378 7b00 ..{..........x{. 0x0040: 0002 c0a8 145c 6cdd ee00 0002 e8bd d101 .....\l......... 0x0050: 77ec 001e 0000 w.....



Next: debug if KVM host doesn't forward some packets

liangbin-pub commented 3 years ago

Complete ARP-Ping-OAM round trip test passed!

Pseudo controller test result:

Command for ping: ping -I 123.0.0.2 -c1 123.0.0.1
executing command: ping -I 123.0.0.2 -c1 123.0.0.1
Output: PING 123.0.0.1 (123.0.0.1) from 123.0.0.2 : 56(84) bytes of data.
64 bytes from 123.0.0.1: icmp_seq=1 ttl=64 time=2.33 ms

--- 123.0.0.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.337/2.337/2.337/0.000 ms

Ping requester (ACA parent) packet capture:

23:46:50.931939 e8:bd:d1:01:77:ec > 82:83:c0:a8:14:6c, ethertype IPv4 (0x0800), length 92: 192.168.20.92.42216 > 192.168.20.108.4789: VXLAN, flags [I] (0x08), vni 888
6c:dd:ee:00:00:02 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 123.0.0.1 tell 123.0.0.2, length 28
    0x0000:  8283 c0a8 146c e8bd d101 77ec 0800 4500  .....l....w...E.
    0x0010:  004e cb0c 4000 4011 c579 c0a8 145c c0a8  .N..@.@..y...\..
    0x0020:  146c a4e8 12b5 003a 0000 0800 0000 0003  .l.....:........
    0x0030:  7800 ffff ffff ffff 6cdd ee00 0002 0806  x.......l.......
    0x0040:  0001 0800 0604 0001 6cdd ee00 0002 7b00  ........l.....{.
    0x0050:  0002 0000 0000 0000 7b00 0001            ........{...
23:46:50.932171 82:83:c0:a8:14:6c > e8:bd:d1:01:77:ec, ethertype IPv4 (0x0800), length 92: 192.168.20.108.42216 > 192.168.20.92.4789: VXLAN, flags [I] (0x08), vni 888
6c:dd:ee:00:00:01 > 6c:dd:ee:00:00:02, ethertype ARP (0x0806), length 42: Reply 123.0.0.1 is-at 6c:dd:ee:00:00:01, length 28
    0x0000:  e8bd d101 77ec 8283 c0a8 146c 0800 4500  ....w......l..E.
    0x0010:  004e cb0c 4000 3f11 c679 c0a8 146c c0a8  .N..@.?..y...l..
    0x0020:  145c a4e8 12b5 003a 0000 0800 0000 0003  .\.....:........
    0x0030:  7800 6cdd ee00 0002 6cdd ee00 0001 0806  x.l.....l.......
    0x0040:  0001 0800 0604 0002 6cdd ee00 0001 7b00  ........l.....{.
    0x0050:  0001 6cdd ee00 0002 7b00 0002            ..l.....{...
23:46:50.932551 e8:bd:d1:01:77:ec > 82:83:c0:a8:14:6e, ethertype IPv4 (0x0800), length 148: 192.168.20.92.36828 > 192.168.20.110.4789: VXLAN, flags [I] (0x08), vni 888
6c:dd:ee:00:00:02 > 6c:dd:ee:00:00:01, ethertype IPv4 (0x0800), length 98: 123.0.0.2 > 123.0.0.1: ICMP echo request, id 23422, seq 1, length 64
    0x0000:  8283 c0a8 146e e8bd d101 77ec 0800 4500  .....n....w...E.
    0x0010:  0086 0683 4000 4011 89c9 c0a8 145c c0a8  ....@.@......\..
    0x0020:  146e 8fdc 12b5 0072 0000 0800 0000 0003  .n.....r........
    0x0030:  7800 6cdd ee00 0001 6cdd ee00 0002 0800  x.l.....l.......
    0x0040:  4500 0054 8729 4000 4001 bd7c 7b00 0002  E..T.)@.@..|{...
    0x0050:  7b00 0001 0800 a81f 5b7e 0001 eaf6 ff5f  {.......[~....._
    0x0060:  0000 0000 3d37 0e00 0000 0000 1011 1213  ....=7..........
    0x0070:  1415 1617 1819 1a1b 1c1d 1e1f 2021 2223  .............!"#
    0x0080:  2425 2627 2829 2a2b 2c2d 2e2f 3031 3233  $%&'()*+,-./0123
    0x0090:  3435 3637                                4567
23:46:50.932999 82:83:c0:a8:14:6e > e8:bd:d1:01:77:ec, ethertype IPv4 (0x0800), length 86: 192.168.20.110.0 > 192.168.20.92.8300: UDP, length 44
    0x0000:  e8bd d101 77ec 8283 c0a8 146e 0800 4500  ....w......n..E.
    0x0010:  0048 d431 0000 3f11 fd58 c0a8 146e c0a8  .H.1..?..X...n..
    0x0020:  145c 0000 206c 0034 0000 0000 0000 7b00  .\...l.4......{.
    0x0030:  0002 7b00 0001 0000 0000 0100 0378 7b00  ..{..........x{.
    0x0040:  0001 c0a8 145d 6cdd ee00 0001 e8bd d101  .....]l.........
    0x0050:  72c8 001e 0000                           r.....
23:46:50.933239 e8:bd:d1:01:72:c8 > 82:83:c0:a8:14:66, ethertype IPv4 (0x0800), length 92: 192.168.20.93.38768 > 192.168.20.102.4789: VXLAN, flags [I] (0x08), vni 888
6c:dd:ee:00:00:01 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 123.0.0.2 tell 123.0.0.1, length 28
    0x0000:  8283 c0a8 1466 e8bd d101 72c8 0800 4500  .....f....r...E.
    0x0010:  004e b58e 4000 4011 dafc c0a8 145d c0a8  .N..@.@......]..
    0x0020:  1466 9770 12b5 003a 0000 0800 0000 0003  .f.p...:........
    0x0030:  7800 ffff ffff ffff 6cdd ee00 0001 0806  x.......l.......
    0x0040:  0001 0800 0604 0001 6cdd ee00 0001 7b00  ........l.....{.
    0x0050:  0001 0000 0000 0000 7b00 0002            ........{...
23:46:50.933823 82:83:c0:a8:14:6e > e8:bd:d1:01:77:ec, ethertype IPv4 (0x0800), length 148: 192.168.20.110.46639 > 192.168.20.92.4789: VXLAN, flags [I] (0x08), vni 888
6c:dd:ee:00:00:01 > 6c:dd:ee:00:00:02, ethertype IPv4 (0x0800), length 98: 123.0.0.1 > 123.0.0.2: ICMP echo reply, id 23422, seq 1, length 64
    0x0000:  e8bd d101 77ec 8283 c0a8 146e 0800 4500  ....w......n..E.
    0x0010:  0086 a47f 4000 3f11 eccc c0a8 146e c0a8  ....@.?......n..
    0x0020:  145c b62f 12b5 0072 0000 0800 0000 0003  .\./...r........
    0x0030:  7800 6cdd ee00 0002 6cdd ee00 0001 0800  x.l.....l.......
    0x0040:  4500 0054 cbac 0000 4001 b8f9 7b00 0001  E..T....@...{...
    0x0050:  7b00 0002 0000 b01f 5b7e 0001 eaf6 ff5f  {.......[~....._
    0x0060:  0000 0000 3d37 0e00 0000 0000 1011 1213  ....=7..........
    0x0070:  1415 1617 1819 1a1b 1c1d 1e1f 2021 2223  .............!"#
    0x0080:  2425 2627 2829 2a2b 2c2d 2e2f 3031 3233  $%&'()*+,-./0123
    0x0090:  3435 3637                                4567

br-tun flow table dump after test should have dynamic flow added by OAM packet (4th packet above), Eric and Minli please check:

Dump flow after PING:
executing command: sudo ovs-ofctl dump-flows br-tun
Output: Huawei@2018
[sudo] password for user:
 cookie=0x0, duration=1.256s, table=0, n_packets=10, n_bytes=938, priority=1,in_port="patch-int" actions=resubmit(,2)
 cookie=0x0, duration=1.213s, table=0, n_packets=2, n_bytes=140, priority=25,in_port="vxlan-generic" actions=resubmit(,4)
 cookie=0x0, duration=1.271s, table=0, n_packets=0, n_bytes=0, priority=0 actions=NORMAL
 cookie=0x0, duration=1.237s, table=2, n_packets=1, n_bytes=42, priority=25,arp,in_port="patch-int",arp_op=1 actions=resubmit(,51)
 cookie=0x0, duration=1.229s, table=2, n_packets=1, n_bytes=98, priority=25,icmp,in_port="patch-int",icmp_type=8 actions=resubmit(,52)
 cookie=0x0, duration=1.251s, table=2, n_packets=0, n_bytes=0, priority=1,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20)
 cookie=0x0, duration=1.246s, table=2, n_packets=8, n_bytes=798, priority=1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,22)
 cookie=0x0, duration=1.186s, table=4, n_packets=2, n_bytes=140, priority=1,tun_id=0x378 actions=mod_vlan_vid:1,output:"patch-int"
 cookie=0x0, duration=1.241s, table=20, n_packets=1, n_bytes=98, priority=1 actions=resubmit(,22)
 cookie=0x0, duration=1.150s, table=22, n_packets=9, n_bytes=884, priority=50,dl_vlan=1 actions=strip_vlan,load:0x378->NXM_NX_TUN_ID[],group:1
 cookie=0x0, duration=1.233s, table=51, n_packets=1, n_bytes=42, priority=1 actions=resubmit(,22)
 cookie=0x0, duration=1.225s, table=52, n_packets=1, n_bytes=98, priority=1 actions=resubmit(,20)

zhangml commented 3 years ago

@liangbin-pub ACA should install the rules on table20 after receiving oam packets, but I did not see it. Please check whether the format of some data packets defined in aca_zeta_oam_server.h is consistent with the format of oam packets sent by zeta to ACA.

struct flow_inject_msg {
  struct in_addr inner_src_ip; // Inner Packet SIP 32bit
  struct in_addr inner_dst_ip; // Inner Packet DIP 32bit
  uint16_t src_port; // Inner Packet SPort  16bit
  uint16_t dst_port; // Inner Packet DPort 16bit
  uint8_t proto; // Inner Packet Protocol 8bit
  uint8_t vni[3]; // tunnel_id, 3bytes 24bit
  struct in_addr inst_dst_ip; // Destination Inst IP 32bit
  struct in_addr node_dst_ip; // Destination Node IP 32bit
  uint8_t inst_dst_mac[6]; // Destination Inst MAC 48bit
  uint8_t node_dst_mac[6]; // Destination Node MAC 48bit
  uint16_t idle_timeout; // 0 - 65536s 16bit
};

er1cthe0ne commented 3 years ago

we got ping working E2E now, the remaining issues are tracked by: #194 and #195

futurewei-cloud / alcor-control-agent

[Zeta] Data path integration #189