Failed to configure LAN-side director IP with keepalived.conf within NAT-mode

[andrewhit]

Here is my network topology in NAT-mode

Client --- Router --- (dpdk0: 192.168.1.2/30) DPVS (dpdk1: 10.10.127.66/26) --- RS (10.10.127.67/26)

Keepalived.conf

static_routes {
     10.10.127.66/26  dev dpdk1
     10.10.127.0/26    via 10.10.127.65
}

local_address_group laddr_g1 {
    10.10.127.66/26 dpdk1  
}

virtual_server_group vip_g1 {
    10.10.76.66       8080
}

virtual_server group vip_g1 {
    delay_loop 3
    lb_algo rr
    lb_kind FNAT
    protocol TCP

    real_server 10.10.127.67 80 {
        weight 100
        inhibit_on_failure
    }
}

root@n10-023-013:/var/log# dpip addr show // it shows 10.10.127.66 is not configured on dpdk1 root@n10-023-013:/var/log# dpip neigh show // there is no ARP entry.

From the below-debugging info, ifa_list is empty so no arp request is sent out.

Sep 12 18:33:45 n10-023-013 ttgw[108161]: NEIGHBOUR: [neigh_arp_confirm] daddr 10.10.127.67
Sep 12 18:33:45 n10-023-013 ttgw[108161]: IFA: inet_addr_select: idev 0x7f33a25ed800, port dpdk1, ifa_cnt 0, list is empty
Sep 12 18:33:45 n10-023-013 ttgw[108161]: NEIGHBOUR: [neigh_arp_confirm] daddr 10.10.127.67
Sep 12 18:33:45 n10-023-013 ttgw[108161]: IFA: inet_addr_select: idev 0x7f33a25ed800, port dpdk1, ifa_cnt 0, list is empty
Sep 12 18:33:45 n10-023-013 ttgw[108161]: NEIGHBOUR: [neigh_arp_confirm]no source ip
Sep 12 18:33:45 n10-023-013 ttgw[108161]: NEIGHBOUR: [neigh_arp_confirm] saddr 0.0.0.0

[andrewhit]

虽然用dpip addr add 10.10.10.127.66/26 dev dpdk1，上面的调试信息显示如下：

Sep 12 18:45:42 n10-023-013 ttgw[113608]: IPVS: conn lookup: [3] TCP 10.2.196.1:49453 -> 10.10.76.66:8080 miss
Sep 12 18:45:42 n10-023-013 ttgw[113608]: IPVS: new conn:  [3] TCP 10.2.196.1:49453 10.10.76.66:8080 10.2.196.1:49453 10.10.127.67:80 refs 2
Sep 12 18:45:42 n10-023-013 ttgw[113608]: IPVS: state trans: TCP in [S...] 10.2.196.1:49453->10.10.127.67:80  state NONE->SYN_RECV conn.refcnt 2
Sep 12 18:45:42 n10-023-013 ttgw[113608]: NEIGHBOUR: [neigh_arp_confirm] daddr 10.10.127.67
Sep 12 18:45:42 n10-023-013 ttgw[113608]: IFA: inet_addr_select: idev 0x7f93a25ed800, port dpdk1, ifa_cnt 1, list is not empty
Sep 12 18:45:42 n10-023-013 ttgw[113608]: IFA: inet_addr_select (1000): flags 80, scope 0, plen 26, addr 10.10.127.66
Sep 12 18:45:42 n10-023-013 ttgw[113608]: IFA: inet_addr_select (2000): flags 80, scope 0, plen 26, addr 10.10.127.66
Sep 12 18:45:42 n10-023-013 ttgw[113608]: NEIGHBOUR: [neigh_arp_confirm] saddr 10.10.127.66
Sep 12 18:45:43 n10-023-013 ttgw[113608]: IPVS: conn lookup: [3] TCP 10.2.196.1:49453 -> 10.10.76.66:8080 hit
Sep 12 18:45:44 n10-023-013 ttgw[113608]: IPVS: conn lookup: [3] TCP 10.2.196.1:49453 -> 10.10.76.66:8080 hit
Sep 12 18:45:45 n10-023-013 ttgw[113608]: IPVS: conn lookup: [3] TCP 10.2.196.1:49453 -> 10.10.76.66:8080 hit
Sep 12 18:45:46 n10-023-013 ttgw[113608]: IPVS: conn lookup: [3] TCP 10.2.196.1:49453 -> 10.10.76.66:8080 hit
Sep 12 18:45:46 n10-023-013 ttgw[113608]: NEIGHBOUR: [neigh_arp_confirm] daddr 10.10.127.67
Sep 12 18:45:46 n10-023-013 ttgw[113608]: IFA: inet_addr_select: idev 0x7f93a25ed800, port dpdk1, ifa_cnt 1, list is not empty
Sep 12 18:45:46 n10-023-013 ttgw[113608]: IFA: inet_addr_select (1000): flags 80, scope 0, plen 26, addr 10.10.127.66
Sep 12 18:45:46 n10-023-013 ttgw[113608]: IFA: inet_addr_select (2000): flags 80, scope 0, plen 26, addr 10.10.127.66
Sep 12 18:45:46 n10-023-013 ttgw[113608]: NEIGHBOUR: [neigh_arp_confirm] saddr 10.10.127.66

但仍然没有学习到10.10.127.67的MAC地址信息

root@n10-023-013:/var/log# dpip neigh show

[andrewhit]

If we choose local_address_group { } to add the static address on some interface, the below two conditions need to be met as described in init_service_vs(virtual_server_t *vs).

    vs->loadbalancing_kind == IP_VS_CONN_F_FULLNAT && vs->local_addr_gname
    ipvs_cmd(LVS_CMD_ADD_LADDR, check_data->vs_group, vs, NULL);

1) lb_kind is FNAT 2) local_group_name laddr_g1 is added on virtual_server group vip_g1.

local_address_group laddr_g1 {
     10.10.127.72 dpdk1  # use DPDK interface
 }

virtual_server group vip_g1 {
     lb_algo rr
     lb_kind FNAT
     protocol TCP
     laddr_group_name laddr_g1
     . . .
}

Thus in nat mode, the interface address fails to be added on dpdk1.

I am also checking another option, static_ipaddress { } to see if it works.

[andrewhit]

https://github.com/iqiyi/dpvs/pull/277