Open w13915984028 opened 2 years ago
**A small tricky:
VIP is allocated from KVM using MAC6a:10:d2:c2:6a:3b
of vip-7e465ab1@harvester-mgmt
, but it is bindded to harvester-mgmt
**
rancher@harv1412:~> ip link
2: ens3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master harvester-mgmt state UP mode DEFAULT group default qlen 1000
link/ether 52:54:00:ae:e9:e2 brd ff:ff:ff:ff:ff:ff
altname enp0s3
3: harvester-mgmt: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 52:54:00:ae:e9:e2 brd ff:ff:ff:ff:ff:ff
65: vip-7e465ab1@harvester-mgmt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
link/ether 6a:10:d2:c2:6a:3b brd ff:ff:ff:ff:ff:ff
rancher@harv1412:~> ip addr
2: ens3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master harvester-mgmt state UP group default qlen 1000
link/ether 52:54:00:ae:e9:e2 brd ff:ff:ff:ff:ff:ff
altname enp0s3
3: harvester-mgmt: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 52:54:00:ae:e9:e2 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.223/24 brd 192.168.122.255 scope global harvester-mgmt
valid_lft forever preferred_lft forever
inet 192.168.122.97/32 scope global harvester-mgmt
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:feae:e9e2/64 scope link
valid_lft forever preferred_lft forever
65: vip-7e465ab1@harvester-mgmt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 6a:10:d2:c2:6a:3b brd ff:ff:ff:ff:ff:ff
inet6 fe80::6810:d2ff:fec2:6a3b/64 scope link
valid_lft forever preferred_lft forever
When check from another VM, the Harvester node IP, VIP has same MAC 52:54:00:ae:e9:e2
(of node VM)
—- ubuntu VM —-
rancher@ubuntuvmday0:~$ ping 192.168.122.97
PING 192.168.122.97 (192.168.122.97) 56(84) bytes of data.
64 bytes from 192.168.122.97: icmp_seq=1 ttl=64 time=1.02 ms
64 bytes from 192.168.122.97: icmp_seq=2 ttl=64 time=0.495 ms
rancher@ubuntuvmday0:~$ ip neigh
192.168.122.97 dev enp1s0 lladdr 52:54:00:ae:e9:e2 REACHABLE
rancher@ubuntuvmday0:~$
rancher@ubuntuvmday0:~$ ping 192.168.122.223
PING 192.168.122.223 (192.168.122.223) 56(84) bytes of data.
64 bytes from 192.168.122.223: icmp_seq=1 ttl=64 time=0.492 ms
rancher@ubuntuvmday0:~$ ip neigh
192.168.122.223 dev enp1s0 lladdr 52:54:00:ae:e9:e2 DELAY
192.168.122.97 dev enp1s0 lladdr 52:54:00:ae:e9:e2 STALE
The VIP is not renewed from KVM DHCP server, it is timed out.
VIP is still pingable, maybe it is because the MAC address is in HOST OS table, but when the MAC is also aged, maybe the VIP is finally lost.
The VIP is seeing aged from KVM DHCP list.
The VIP is allocated from KVM using MAC 6a:10:d2:c2:6a:3b of vip-7e465ab1@harvester-mgmt, but it is attached to harvester-mgmt
It looks, when the Harvester goes into "ready" state, the VIP is last re-leased from DHCP server, after that, no renewing.
Which module is responsible for renewing the VIP? If it try to get VIP (ipv4) from dev vip-7e465ab1@harvester-mgmt
, then it will fail, the VIP is attached to dev harvester-mgmt
.
consider enhancement https://github.com/harvester/harvester/issues/1683
Even when keep the hostname as "rancher", it looks the NODE IP from KVM still changed, needs further investigation
Even when keep the hostname as "rancher", it looks the NODE IP from KVM still changed, needs further investigation
The "Client ID or DUID" has a change, we did not notice it before
ff:00:f7:72:bf:00:01:00:01:29:4e_:04:bd_:52:54:00:f7:72:bf
ff:00:f7:72:bf:00:01:00:01:29:4e:_06:ff_:52:54:00:f7:72:bf
virsh # net-dhcp-leases --network default
Expiry Time MAC address Protocol IP address Hostname Client ID or DUID
----------------------------------------------------------------------------------------------------------------------------------------------------
2021-12-16 10:23:11 46:6c:14:89:e8:ee ipv4 192.168.122.225/24 - -
2021-12-16 10:08:14 52:54:00:f7:72:bf ipv4 192.168.122.71/24 rancher ff:00:f7:72:bf:00:01:00:01:29:4e:04:bd:52:54:00:f7:72:bf
2021-12-16 10:17:38 52:54:00:f7:72:bf ipv4 192.168.122.72/24 rancher ff:00:f7:72:bf:00:01:00:01:29:4e:06
Reproduce the issue: https://github.com/harvester/harvester/issues/1668
two IPs are allocated from KVM embedded dhcp server, 192.168.122.222 is node IP, 192.168.222.97 is VIP
Harvester installation![image](https://user-images.githubusercontent.com/31133476/146020278-65044071-90b2-41c2-9fe8-81bb39ffd764.png)
Harvester node reboot after installation
Harvester VIP is ready now:![image](https://user-images.githubusercontent.com/31133476/146021108-71a8e55a-935c-499b-8780-02762e5a2929.png)
VIP is newed![image](https://user-images.githubusercontent.com/31133476/146021425-cc3e58e9-0049-4395-b12a-3b5e09167b7b.png)