search

tsujamin / hass-addons

108 stars 35 forks source link

Strange problem after last update #71

Closed matata86 closed 1 year ago

matata86 commented 1 year ago

Hi, after the last update of the addon I'm solving a strange problem. I have two instances of home assistant connected to tailscale. Both instances report tailscale as online. Machines-Tailscale

If I give the ping command on my laptop, which is also connected to the VPN tailscale, everything is ok. terminal

But if I want to get data from another instance via rest api, it doesn't work. I have been using it like this for many months and everything worked. But now even from the HA terminal the instances can't see each other and don't communicate.

terminal_ha

Log from addon here:

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
2023/03/11 16:14:06 logtail started
2023/03/11 16:14:06 Program starting: v1.36.2-t9450812f7-g622a25149, Go 1.19.4-tsdc0ce6324d: []string{"tailscaled", "-cleanup", "-statedir", "/data", "-state", "/data/tailscaled.state", "-socket", "/var/run/tailscale/tailscaled.sock", "--tun=userspace-networking"}
2023/03/11 16:14:06 LogID: b28eb118eb01b43007dd0d657fea89750756541552f46aabb1801e4eb28e3842
2023/03/11 16:14:06 logpolicy: using system state directory "/var/lib/tailscale"
logpolicy.ConfigFromFile /var/lib/tailscale/tailscaled.log.conf: open /var/lib/tailscale/tailscaled.log.conf: no such file or directory
logpolicy.Config.Validate for /var/lib/tailscale/tailscaled.log.conf: config is nil
2023/03/11 16:14:06 dns: [rc=unknown ret=direct]
2023/03/11 16:14:06 dns: using "direct" mode
2023/03/11 16:14:06 dns: using *dns.directManager
2023/03/11 16:14:06 flushing log.
2023/03/11 16:14:06 logger closing down
tailscaled not started yet, sleeping 5s
2023/03/11 16:14:06 logtail started
2023/03/11 16:14:06 Program starting: v1.36.2-t9450812f7-g622a25149, Go 1.19.4-tsdc0ce6324d: []string{"tailscaled", "-statedir", "/data", "-state", "/data/tailscaled.state", "-socket", "/var/run/tailscale/tailscaled.sock", "--tun=userspace-networking"}
2023/03/11 16:14:06 LogID: b28eb118eb01b43007dd0d657fea89750756541552f46aabb1801e4eb28e3842
2023/03/11 16:14:06 logpolicy: using system state directory "/var/lib/tailscale"
2023/03/11 16:14:06 wgengine.NewUserspaceEngine(tun "userspace-networking") ...
2023/03/11 16:14:06 dns: using dns.noopManager
2023/03/11 16:14:06 link state: interfaces.State{defaultRoute=eno1 ifs={docker0:[172.30.232.1/23] eno1:[192.168.1.10/24] hassio:[172.30.32.1/23]} v4=true v6=false}
2023/03/11 16:14:06 magicsock: disco key = d:04815ae7e297a6db
2023/03/11 16:14:06 Creating WireGuard device...
2023/03/11 16:14:06 Bringing WireGuard device up...
2023/03/11 16:14:06 Bringing router up...
2023/03/11 16:14:06 Clearing router settings...
2023/03/11 16:14:06 Starting link monitor...
2023/03/11 16:14:06 Engine created.
2023/03/11 16:14:06 pm: using backend prefs for "profile-07e0": Prefs{ra=false dns=true want=true routes=[] nf=on host="HA-home" Persist{lm=, o=, n=[gL6x3] u="ma.prochazka@gmail.com"}}
2023/03/11 16:14:06 got LocalBackend in 20ms
2023/03/11 16:14:06 Start
2023/03/11 16:14:06 Backend: logs: be:b28eb118eb01b43007dd0d657fea89750756541552f46aabb1801e4eb28e3842 fe:
2023/03/11 16:14:06 control: client.Login(false, 0)
2023/03/11 16:14:06 control: doLogin(regen=false, hasUrl=false)
2023/03/11 16:14:06 health("overall"): error: not in map poll
2023/03/11 16:14:06 control: control server key from https://controlplane.tailscale.com: ts2021=[fSeS+], legacy=[nlFWp]
2023/03/11 16:14:06 control: RegisterReq: onode= node=[gL6x3] fup=false nks=false
2023/03/11 16:14:06 control: creating new noise client
2023/03/11 16:14:06 control: RegisterReq: got response; nodeKeyExpired=false, machineAuthorized=true; authURL=false
2023/03/11 16:14:06 control: netmap: got new dial plan from control
2023/03/11 16:14:06 active login: ma.prochazka@gmail.com
2023/03/11 16:14:06 Switching ipn state NoState -> Starting (WantRunning=true, nm=true)
2023/03/11 16:14:06 magicsock: SetPrivateKey called (init)
2023/03/11 16:14:06 wgengine: Reconfig: configuring userspace WireGuard config (with 0/6 peers)
2023/03/11 16:14:06 wgengine: Reconfig: configuring router
2023/03/11 16:14:06 wgengine: Reconfig: configuring DNS
2023/03/11 16:14:06 dns: Set: {DefaultResolvers:[] Routes:{} SearchDomains:[] Hosts:7}
2023/03/11 16:14:06 dns: Resolvercfg: {Routes:{} Hosts:7 LocalDomains:[]}
2023/03/11 16:14:06 dns: OScfg: {Nameservers:[] SearchDomains:[] MatchDomains:[] Hosts:[]}
2023/03/11 16:14:06 peerapi: serving on http://100.94.XXX:39385
2023/03/11 16:14:06 peerapi: serving on http://[fd7a:115c:a1e0:ab12:4843:cd96:625e:bf41]:39385
2023/03/11 16:14:06 portmapper: UPnP meta changed: {Location:http://192.168.1.1:1900/qyyjn/rootDesc.xml Server:TP-Link/TP-Link UPnP/1.1 MiniUPnPd/1.8 USN:uuid:e4c797e0-82ce-4f84-b56d-085d9d77c8cd::urn:schemas-upnp-org:device:InternetGatewayDevice:1}
2023/03/11 16:14:07 magicsock: home is now derp-4 (fra)
2023/03/11 16:14:07 magicsock: adding connection to derp-4 for home-keep-alive
2023/03/11 16:14:07 magicsock: 1 active derp conns: derp-4=cr0s,wr0s
2023/03/11 16:14:07 Switching ipn state Starting -> Running (WantRunning=true, nm=true)
2023/03/11 16:14:07 control: NetInfo: NetInfo{varies=false hairpin=false ipv6=false ipv6os=true udp=true icmpv4=false derp=#4 portmap=active-UM link=""}
2023/03/11 16:14:07 derphttp.Client.Connect: connecting to derp-4 (fra)
2023/03/11 16:14:07 magicsock: endpoints changed: 10.10.22.192:34829 (portmap), 62.201.31.228:34829 (stun), 172.30.32.1:34829 (local), 172.30.232.1:34829 (local), 192.168.1.10:34829 (local)
2023/03/11 16:14:07 control: netmap: got new dial plan from control
2023/03/11 16:14:07 magicsock: derp-4 connected; connGen=1
2023/03/11 16:14:07 health("overall"): ok
2023/03/11 16:14:11 Start
2023/03/11 16:14:11 control: client.Shutdown()
2023/03/11 16:14:11 control: client.Shutdown: inSendStatus=0
2023/03/11 16:14:11 active login: 
2023/03/11 16:14:11 control: mapRoutine: quit
2023/03/11 16:14:11 control: Client.Shutdown done.
2023/03/11 16:14:11 control: NetInfo: NetInfo{varies=false hairpin=false ipv6=false ipv6os=true udp=true icmpv4=false derp=#4 portmap=active-UM link=""}
2023/03/11 16:14:11 Backend: logs: be:b28eb118eb01b43007dd0d657fea89750756541552f46aabb1801e4eb28e3842 fe:
2023/03/11 16:14:11 control: client.Login(false, 0)
2023/03/11 16:14:11 control: doLogin(regen=false, hasUrl=false)
2023/03/11 16:14:11 control: control server key from https://controlplane.tailscale.com: ts2021=[fSeS+], legacy=[nlFWp]
2023/03/11 16:14:11 control: RegisterReq: onode= node=[gL6x3] fup=false nks=false
2023/03/11 16:14:11 control: creating new noise client
2023/03/11 16:14:11 control: RegisterReq: got response; nodeKeyExpired=false, machineAuthorized=true; authURL=false
2023/03/11 16:14:12 control: netmap: got new dial plan from control
2023/03/11 16:14:12 active login: ma.prochazka@gmail.com
2023/03/11 16:14:12 Switching ipn state NoState -> Starting (WantRunning=true, nm=true)
2023/03/11 16:14:12 Switching ipn state Starting -> Running (WantRunning=true, nm=true)
tailscaled ${TAILSCALED_FLAGS[@]}
2023/03/11 19:29:34 health("overall"): error: not in map poll
2023/03/11 19:29:34 control: netmap: got new dial plan from control
2023/03/11 19:29:34 health("overall"): ok
2023/03/11 21:04:34 wgengine: idle peer [aoeYz] now active, reconfiguring WireGuard
2023/03/11 21:04:34 wgengine: Reconfig: configuring userspace WireGuard config (with 1/6 peers)
2023/03/11 21:04:34 magicsock: disco: node [aoeYz] d:7b5db0acdff97601 now using 192.168.1.150:41641
2023/03/11 21:04:35 Accept: ICMPv4{100.85.XXX:0 > 100.94.XXX:0} 84 icmp ok
2023/03/11 21:04:35 Accept: ICMPv4{100.85.XXX:0 > 100.94.XXX:0} 84 icmp ok
2023/03/11 21:04:36 Accept: ICMPv4{100.85.XXX:0 > 100.94.XXX:0} 84 icmp ok
2023/03/11 21:23:36 wgengine: Reconfig: configuring userspace WireGuard config (with 0/6 peers)

Do you have any idea where the problem might be?

tsujamin commented 1 year ago

Hey mate Not sure, only thing I can think is that I defaulted on userspace-networking per another issue raised. Maybe toggle that?On 12 Mar 2023, at 8:38 am, matata86 @.***> wrote: Hi, after the last update of the addon I'm solving a strange problem. I have two instances of home assistant connected to tailscale. Both instances report tailscale as online.

If I give the ping command on my laptop, which is also connected to the VPN tailscale, everything is ok.

But if I want to get data from another instance via rest api, it doesn't work. I have been using it like this for many months and everything worked. But now even from the HA terminal the instances can't see each other and don't communicate.

Log from addon here: s6-rc: info: service s6rc-oneshot-runner: starting s6-rc: info: service s6rc-oneshot-runner successfully started s6-rc: info: service fix-attrs: starting s6-rc: info: service fix-attrs successfully started s6-rc: info: service legacy-cont-init: starting s6-rc: info: service legacy-cont-init successfully started s6-rc: info: service legacy-services: starting s6-rc: info: service legacy-services successfully started 2023/03/11 16:14:06 logtail started 2023/03/11 16:14:06 Program starting: v1.36.2-t9450812f7-g622a25149, Go 1.19.4-tsdc0ce6324d: []string{"tailscaled", "-cleanup", "-statedir", "/data", "-state", "/data/tailscaled.state", "-socket", "/var/run/tailscale/tailscaled.sock", "--tun=userspace-networking"} 2023/03/11 16:14:06 LogID: b28eb118eb01b43007dd0d657fea89750756541552f46aabb1801e4eb28e3842 2023/03/11 16:14:06 logpolicy: using system state directory "/var/lib/tailscale" logpolicy.ConfigFromFile /var/lib/tailscale/tailscaled.log.conf: open /var/lib/tailscale/tailscaled.log.conf: no such file or directory logpolicy.Config.Validate for /var/lib/tailscale/tailscaled.log.conf: config is nil 2023/03/11 16:14:06 dns: [rc=unknown ret=direct] 2023/03/11 16:14:06 dns: using "direct" mode 2023/03/11 16:14:06 dns: using *dns.directManager 2023/03/11 16:14:06 flushing log. 2023/03/11 16:14:06 logger closing down tailscaled not started yet, sleeping 5s 2023/03/11 16:14:06 logtail started 2023/03/11 16:14:06 Program starting: v1.36.2-t9450812f7-g622a25149, Go 1.19.4-tsdc0ce6324d: []string{"tailscaled", "-statedir", "/data", "-state", "/data/tailscaled.state", "-socket", "/var/run/tailscale/tailscaled.sock", "--tun=userspace-networking"} 2023/03/11 16:14:06 LogID: b28eb118eb01b43007dd0d657fea89750756541552f46aabb1801e4eb28e3842 2023/03/11 16:14:06 logpolicy: using system state directory "/var/lib/tailscale" 2023/03/11 16:14:06 wgengine.NewUserspaceEngine(tun "userspace-networking") ... 2023/03/11 16:14:06 dns: using dns.noopManager 2023/03/11 16:14:06 link state: interfaces.State{defaultRoute=eno1 ifs={docker0:[172.30.232.1/23] eno1:[192.168.1.10/24] hassio:[172.30.32.1/23]} v4=true v6=false} 2023/03/11 16:14:06 magicsock: disco key = d:04815ae7e297a6db 2023/03/11 16:14:06 Creating WireGuard device... 2023/03/11 16:14:06 Bringing WireGuard device up... 2023/03/11 16:14:06 Bringing router up... 2023/03/11 16:14:06 Clearing router settings... 2023/03/11 16:14:06 Starting link monitor... 2023/03/11 16:14:06 Engine created. 2023/03/11 16:14:06 pm: using backend prefs for "profile-07e0": Prefs{ra=false dns=true want=true routes=[] nf=on host="HA-home" Persist{lm=, o=, n=[gL6x3] @."}} 2023/03/11 16:14:06 got LocalBackend in 20ms 2023/03/11 16:14:06 Start 2023/03/11 16:14:06 Backend: logs: be:b28eb118eb01b43007dd0d657fea89750756541552f46aabb1801e4eb28e3842 fe: 2023/03/11 16:14:06 control: client.Login(false, 0) 2023/03/11 16:14:06 control: doLogin(regen=false, hasUrl=false) 2023/03/11 16:14:06 health("overall"): error: not in map poll 2023/03/11 16:14:06 control: control server key from https://controlplane.tailscale.com: ts2021=[fSeS+], legacy=[nlFWp] 2023/03/11 16:14:06 control: RegisterReq: onode= node=[gL6x3] fup=false nks=false 2023/03/11 16:14:06 control: creating new noise client 2023/03/11 16:14:06 control: RegisterReq: got response; nodeKeyExpired=false, machineAuthorized=true; authURL=false 2023/03/11 16:14:06 control: netmap: got new dial plan from control 2023/03/11 16:14:06 active login: @. 2023/03/11 16:14:06 Switching ipn state NoState -> Starting (WantRunning=true, nm=true) 2023/03/11 16:14:06 magicsock: SetPrivateKey called (init) 2023/03/11 16:14:06 wgengine: Reconfig: configuring userspace WireGuard config (with 0/6 peers) 2023/03/11 16:14:06 wgengine: Reconfig: configuring router 2023/03/11 16:14:06 wgengine: Reconfig: configuring DNS 2023/03/11 16:14:06 dns: Set: {DefaultResolvers:[] Routes:{} SearchDomains:[] Hosts:7} 2023/03/11 16:14:06 dns: Resolvercfg: {Routes:{} Hosts:7 LocalDomains:[]} 2023/03/11 16:14:06 dns: OScfg: {Nameservers:[] SearchDomains:[] MatchDomains:[] Hosts:[]} 2023/03/11 16:14:06 peerapi: serving on http://100.94.XXX:39385 2023/03/11 16:14:06 peerapi: serving on http://[fd7a:115c:a1e0:ab12:4843:cd96:625e:bf41]:39385 2023/03/11 16:14:06 portmapper: UPnP meta changed: {Location:http://192.168.1.1:1900/qyyjn/rootDesc.xml Server:TP-Link/TP-Link UPnP/1.1 MiniUPnPd/1.8 USN:uuid:e4c797e0-82ce-4f84-b56d-085d9d77c8cd::urn:schemas-upnp-org:device:InternetGatewayDevice:1} 2023/03/11 16:14:07 magicsock: home is now derp-4 (fra) 2023/03/11 16:14:07 magicsock: adding connection to derp-4 for home-keep-alive 2023/03/11 16:14:07 magicsock: 1 active derp conns: derp-4=cr0s,wr0s 2023/03/11 16:14:07 Switching ipn state Starting -> Running (WantRunning=true, nm=true) 2023/03/11 16:14:07 control: NetInfo: NetInfo{varies=false hairpin=false ipv6=false ipv6os=true udp=true icmpv4=false derp=#4 portmap=active-UM link=""} 2023/03/11 16:14:07 derphttp.Client.Connect: connecting to derp-4 (fra) 2023/03/11 16:14:07 magicsock: endpoints changed: 10.10.22.192:34829 (portmap), 62.201.31.228:34829 (stun), 172.30.32.1:34829 (local), 172.30.232.1:34829 (local), 192.168.1.10:34829 (local) 2023/03/11 16:14:07 control: netmap: got new dial plan from control 2023/03/11 16:14:07 magicsock: derp-4 connected; connGen=1 2023/03/11 16:14:07 health("overall"): ok 2023/03/11 16:14:11 Start 2023/03/11 16:14:11 control: client.Shutdown() 2023/03/11 16:14:11 control: client.Shutdown: inSendStatus=0 2023/03/11 16:14:11 active login: 2023/03/11 16:14:11 control: mapRoutine: quit 2023/03/11 16:14:11 control: Client.Shutdown done. 2023/03/11 16:14:11 control: NetInfo: NetInfo{varies=false hairpin=false ipv6=false ipv6os=true udp=true icmpv4=false derp=#4 portmap=active-UM link=""} 2023/03/11 16:14:11 Backend: logs: be:b28eb118eb01b43007dd0d657fea89750756541552f46aabb1801e4eb28e3842 fe: 2023/03/11 16:14:11 control: client.Login(false, 0) 2023/03/11 16:14:11 control: doLogin(regen=false, hasUrl=false) 2023/03/11 16:14:11 control: control server key from https://controlplane.tailscale.com: ts2021=[fSeS+], legacy=[nlFWp] 2023/03/11 16:14:11 control: RegisterReq: onode= node=[gL6x3] fup=false nks=false 2023/03/11 16:14:11 control: creating new noise client 2023/03/11 16:14:11 control: RegisterReq: got response; nodeKeyExpired=false, machineAuthorized=true; authURL=false 2023/03/11 16:14:12 control: netmap: got new dial plan from control 2023/03/11 16:14:12 active login: @.*** 2023/03/11 16:14:12 Switching ipn state NoState -> Starting (WantRunning=true, nm=true) 2023/03/11 16:14:12 Switching ipn state Starting -> Running (WantRunning=true, nm=true) tailscaled ${TAILSCALED_FLAGS[@]} 2023/03/11 19:29:34 health("overall"): error: not in map poll 2023/03/11 19:29:34 control: netmap: got new dial plan from control 2023/03/11 19:29:34 health("overall"): ok 2023/03/11 21:04:34 wgengine: idle peer [aoeYz] now active, reconfiguring WireGuard 2023/03/11 21:04:34 wgengine: Reconfig: configuring userspace WireGuard config (with 1/6 peers) 2023/03/11 21:04:34 magicsock: disco: node [aoeYz] d:7b5db0acdff97601 now using 192.168.1.150:41641 2023/03/11 21:04:35 Accept: ICMPv4{100.85.XXX:0 > 100.94.XXX:0} 84 icmp ok 2023/03/11 21:04:35 Accept: ICMPv4{100.85.XXX:0 > 100.94.XXX:0} 84 icmp ok 2023/03/11 21:04:36 Accept: ICMPv4{100.85.XXX:0 > 100.94.XXX:0} 84 icmp ok 2023/03/11 21:23:36 wgengine: Reconfig: configuring userspace WireGuard config (with 0/6 peers) Do you have any idea where the problem might be?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were assigned.Message ID: @.***>

matata86 commented 1 year ago

Hi, thank you for reply. Yes, it worked. I cancelled this option in the settings and it's working properly now. What is the purpose of this option, what are its advantages in general?

tsujamin commented 1 year ago

Awesome glad that fixed it, and sorry it caused a regression for you 🥲

In short, the option causes Tailscale to not "route" incoming and outgoing packets via a virtual networking device (tailscale0) and instead emulates a lot of the outbound routing via normal sockets. The upside is that it works in a lot of scenarios where Tailscale can't fully control the host networking stack, but the downside is some functionality doesn't work quite right.

Over the last 6 or so months more and more issues people had with this add-on were resolved by enabling userspace-networking, probably because in a lot of HomeAssiatant installations add-ons the network stack is managed by the host OS/supervisors. I figured for the benefit of most users it'd make sense to make it the default, I just didn't realise the way I defaulted it in config would affect existing users (sorry!)

Tl;dr if not using userspace-networking works for you keep using it that way and reap the benefits! But it's necessary for a lot of users it appears

tsujamin commented 1 year ago

Let me know if I'm right to close this ticket off

matata86 commented 1 year ago

OK, thank you. Solved!

ovizii commented 1 year ago

I just wanted to let you know that the "userspace networking" option broke my backups via SMB (overtailscale) – unticking that option made everything work again. This is the SMB Backup add-on I am using: https://github.com/thomasmauerer/hassio-addons/tree/master/samba-backup

tsujamin commented 1 year ago

Ah sorry about that

ovizii commented 1 year ago

No worries, just wanted to make it known in case someone else comes here looking to fix the same problem. I ended up here after trying to debug my samba server, the HA instance and the Backup add-on :-)