aleks-mariusz
commented
2 months ago Below, I captured a tcpdump during the incident over the weekend, just of all icmp6 packets (the dhcpv6 traffic itself is not captured) - I am posting two versions, one annotated by myself (with the commentary of what i believe is happening), as well as an unannotated/unabridged version (if you don't want my comments/interpretation of the traffic to bias your investigation).
Note i am using a 'diff-like output' to colorize the different "speakers" of each ICMPv6 packet to make to easier to follow:
The hosts involved and appearing in the below captures are color-coded as follows (along with their IPv6 addresses, both link-local and global-unicast):
+ router-main:bd:10 - provides IPv6 connectivity normally, via SLAAC-enabling router-advertisements proposing the client uses DHCPv6
+ 2a02:aaaa:bbbb:2220:: - my statically assigned address for this VLAN's interface - what the HA vm starts impersonating
+ fe80::c0bf:3704:2f17:8093 - link-local stable-privacy address prior-to--reboot
+ fe80::6c0d:8563:da46:ca9c - link-local stable-privacy address after-reboot
+ fe80::213f:816a:a47b:30e4 - link-local stable-privacy address pre-issue-fixed
+ fe80::33b0:f83f:9c09:f500 - link-local stable-privacy address pre-issue-fixed
+ fe80::9bec:db29:40d6:3f66 - link-local stable-privacy address after-issue-fixed
- server-kvm_:e1:b8 - this is my NAS which is the hypervisor for the home-assistant VM
- fe80::d250:99ff:fe6f:e1b8 - link-local eui64-based address
- 2a02:aaaa:bbbb:2220:d250:99ff:fe6f:e1b8 - slaac-assigned eui64-based address
! vm-homeastn:a5:c6 - this is the actual VM running home-assistant OS
! fe80::a8b4:7a51:21a4:a9ba - link-local eui64-based address
! 2a02:aaaa:bbbb:2220::11 - dhcpv6-assigned global addresss
! 2a02:aaaa:bbbb:2220:6bf3:b959:29ea:d161 - slaac-assigned address
@@ macbook-pro:0c:20 - this is my macbook which is also impacted by this issue ---@@
@@ fe80::1430:f31f:50e3:e9d0 - link local ---@@
@@ 2a02:aaaa:bbbb:2220::1 - DHCPv6 assigned address ---@@
@@ 2a02:aaaa:bbbb:2220:7c31:b848:97e:7840 - SLAAC-assigned privacy address ---@@
# and finally this contains comments w/ my understanding of about what's happening on the following linekey times in the capture
pre-14:00 - everything working normally 14:00:00.000 - the router has been rebooted 14:01:08.391 - the HA vm begins impersonating the router's assigned IPv6 gua address (answering to another host on network, a macbook) 14:01:13.500 - the router has sent its first ICMPv6 message since booting back up 14:01:52.257 - final HA-vm's impersonation of router, at this point it is rebooted. 14:25:04.935 - the router performs it's DAD again, successfully - things from then on continue to work normally. 14:25:06.154 - the HA vm starts coming back up 14:25:26.377 - the HA vm appears to try to impersonate again, but this may have been after the DAD timeout expired
agners
Describe the issue you are experiencing
I've just started playing with HomeAssistant, and so to dip my toes, I created a VM back in July on my NAS (which runs Linux, using libvirtd/kvm - While I haven't done anything with HASS (other than set up a TLS cert, install HACS and the ssh-addon), for the most part I'm pretty new to HA but quite comfortable w/ linux/containers/networking (it's my day job).
I have on my home-network dual-stack (IPv4 + IPv6), and I noticed that my IPv6 connectivity starts to act "funny" ever since around the time i set up HA, which seems was uncoincidental.. I have been having issues for months on my home network related to IPv6 connectivity breaking seemingly "randomly" (i only now just figured out what triggers it).
So of course I logged into my home-router (running OpenWRT) and I noticed the following log messages on my home-router:
What seems to be happening is that the router fails doing its attempt at duplicate-address detection because the mac-address of my HomeAssistant VM is advertising itself as the address the router is trying to use. I've of course double-and-triple-checked that the home-assistant VM is not configured to use that address.
Seems specifically only-triggered whenever i reboot my home-router (so it's something i do a few times a month to keep up to date with patches).. The router boots back up, it suddenly is unable to assign itself the configured IPv6 address, because for some reason, when doing DAD and it asks if anyone has the address, the home-assistant VM starts responding to the router's neighbor-solicitation requests (this use of neighbor solicitation is standard for duplicate-address-detection for any IPv6 host before it assigns itself an address, it asks if anyone else is using it first).
Now i've made sure the home-assistant has been allocated an entirely different static dhcpv6 assignment, and so it's unclear WHY it starts behaving this way once the router goes down. But because my home-router (when it boots back up) can no longer assign itself the IPv6 address to the VLAN interface for the network HA is on, all my hosts on the vlan no longer have working IPv6 due to DHCPv6 stopping working
The only fix is to reboot the HA vm guest.
What i'm aiming to find out is, why is HAOS doing this?
tcpdump captures in the next comment, otherwise i get a max-content error > 65536 characters long)
What operating system image do you use?
generic-x86-64 (Generic UEFI capable x86-64 systems)
What version of Home Assistant Operating System is installed?
Home Assistant OS 13.1
Did the problem occur after upgrading the Operating System?
No
Hardware details
output of `lscpu` on VM
``` # lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 36 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-7 Vendor ID: GenuineIntel BIOS Vendor ID: Red Hat Model name: Westmere E56xx/L56xx/X56xx (IBRS update) BIOS Model name: RHEL-8.6.0 PC (Q35 + ICH9, 2009) CPU @ 2.0GHz BIOS CPU family: 1 CPU family: 6 Model: 44 Thread(s) per core: 1 Core(s) per socket: 1 Socket(s): 8 Stepping: 1 BogoMIPS: 4799.99 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge m ca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes rdrand hype rvisor lahf_lm 3dnowprefetch cpuid_fault pti ibrs ibpb stibp tsc_adjust smep erms arat umip md_clear arch_ca pabilities Virtualization features: Hypervisor vendor: KVM Virtualization type: full Caches (sum of all): L1d: 256 KiB (8 instances) L1i: 256 KiB (8 instances) L2: 32 MiB (8 instances) L3: 128 MiB (8 instances) NUMA: NUMA node(s): 1 NUMA node0 CPU(s): 0-7 Vulnerabilities: Gather data sampling: Not affected Itlb multihit: Not affected L1tf: Mitigation; PTE Inversion Mds: Mitigation; Clear CPU buffers; SMT Host state unknown Meltdown: Mitigation; PTI Mmio stale data: Unknown: No mitigations Reg file data sampling: Not affected Retbleed: Not affected Spec rstack overflow: Not affected Spec store bypass: Not affected Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointe r sanitization Spectre v2: Mitigation; Retpolines; IBPB conditional; IBRS_FW; STI BP disabled; RSB filling; PBRSB-eIBRS Not affected; BH I Retpoline Srbds: Not affected Tsx async abort: Not affected ```output of `free` (memory-info) on VM
``` # free -m total used free shared buff/cache available Mem: 3900 662 1864 1 1373 3175 Swap: 1287 0 1287 ```output of `virsh dumpxml homeassistant` on hypervisor
```Steps to reproduce the issue
Anything in the Supervisor logs that might be useful for us?