search

zerotier / ZeroTierOne

A Smart Ethernet Switch for Earth
https://zerotier.com
Other
14.17k stars 1.65k forks source link

Zerotier 1.6.4: Kernel: netlink: 8 bytes leftover after parsing attributes in process `zerotier-one’ #1380

Open dhbaird opened 3 years ago

dhbaird commented 3 years ago

Required

Additional information

zerotier-one -h
# >>> ZeroTier One version 1.6.4
uname -a
# >>> Linux hello-world 5.4.79-v7l+ #1373 SMP Mon Nov 23 13:27:40 GMT 2020 armv7l GNU/Linux

I noticed some other people experiencing this issue to: https://discuss.zerotier.com/t/kernel-netlink-8-bytes-leftover-after-parsing-attributes-in-process-zerotier-one/1823

Thanks!

dhbaird commented 3 years ago

I'm able to confirm version when issue was introduced: zerotier-one_1.6.4_armhf has issue zerotier-one_1.6.0_armhf has issue zerotier-one_1.4.6_armhf does not have issue

adamierymenko commented 3 years ago

The change was moving to internal netlink instead of invoking /sbin/ip for all network config. Looks like a minor bug in our code somewhere in LinuxNetLink.

oriolrius commented 3 years ago

Same issue with:

# zerotier-one -v
1.6.5
# uname -a
Linux MXE_JC45NS6009 5.4.0-72-generic #80~18.04.1-Ubuntu SMP Mon Apr 12 23:26:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Take a look:

# dmesg | tail
[87448.679009] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[87478.678321] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[87483.091207] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[87508.683166] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[87538.688638] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[87548.307714] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[87568.692967] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[87598.696765] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[87613.437929] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[87628.700935] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.

I don't know why some peers of the network lost connection randomly, restarting their zerotier service everything works again. Those peers are Linux, Mac and Windows boxes.

VectorKappa commented 2 years ago

I can confirm the issue exists on Arch Linux.

❯ zerotier-one -v
1.6.6
❯ uname -a
Linux NULL 5.14.9-zen2-1-zen #1 ZEN SMP PREEMPT Fri, 01 Oct 2021 19:03:23 +0000 x86_64 GNU/Linux
PHCSJC commented 2 years ago

I have test in my device, version 1.6.3-1 is ok, it don't have this issue

erikh commented 2 years ago

it's likely a combination of kernel version and zerotier version, fwiw.

13werwolf13 commented 2 years ago

i faced the same problem on my router

root@OpenWrt:~# dmesg | tail
[ 2708.530851] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[ 2719.642247] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[ 2738.582901] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[ 2768.613569] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[ 2780.049152] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[ 2793.492264] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[ 2798.632233] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[ 2817.197861] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
root@OpenWrt:~# zerotier-one -v
1.6.6
root@OpenWrt:~# uname -a
Linux OpenWrt 5.4.143 #0 SMP Tue Aug 31 22:20:08 2021 mips GNU/Linux
root@OpenWrt:~# cat /etc/openwrt_release 
DISTRIB_ID='OpenWrt'
DISTRIB_RELEASE='21.02.0'
DISTRIB_REVISION='r16279-5cc0535800'
DISTRIB_TARGET='ramips/mt7621'
DISTRIB_ARCH='mipsel_24kc'
DISTRIB_DESCRIPTION='OpenWrt 21.02.0 r16279-5cc0535800'
DISTRIB_TAINTS=''
root@OpenWrt:~# cat /etc/openwrt_version 
r16279-5cc0535800
root@OpenWrt:~# cat /proc/cpuinfo | head -n4
system type             : MediaTek MT7621 ver:1 eco:3
machine                 : Xiaomi Redmi Router AC2100
processor               : 0
cpu model               : MIPS 1004Kc V2.15

and to this router speed from white ip, wg, zerotier:

[werwolf@work] ~  
❯ iperf3 -c WHITEIP
Connecting to host WHITEIP, port 5201
[  5] local XXX.XXX.XXX.XXX port 38002 connected to WHITEIP port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  84.2 MBytes   706 Mbits/sec    0    469 KBytes       
[  5]   1.00-2.00   sec  84.8 MBytes   711 Mbits/sec    0    469 KBytes       
[  5]   2.00-3.00   sec  80.5 MBytes   675 Mbits/sec    0    469 KBytes       
[  5]   3.00-4.00   sec  85.6 MBytes   718 Mbits/sec    0    469 KBytes       
[  5]   4.00-5.00   sec  86.3 MBytes   724 Mbits/sec    0    469 KBytes       
[  5]   5.00-6.00   sec  84.8 MBytes   711 Mbits/sec    0    469 KBytes       
[  5]   6.00-7.00   sec  79.6 MBytes   668 Mbits/sec    0    469 KBytes       
[  5]   7.00-8.00   sec  84.8 MBytes   711 Mbits/sec    0    469 KBytes       
[  5]   8.00-9.00   sec  80.4 MBytes   675 Mbits/sec    0    469 KBytes       
[  5]   9.00-10.00  sec  78.5 MBytes   659 Mbits/sec    0    469 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   829 MBytes   696 Mbits/sec    0             sender
[  5]   0.00-10.00  sec   828 MBytes   694 Mbits/sec                  receiver

[werwolf@home] ~  
❯ iperf3 -c WIREGUARDTUNNELIP
Connecting to host 192.168.200.254, port 5201
[  5] local 192.168.200.8 port 38006 connected to 192.168.200.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  83.0 MBytes   696 Mbits/sec    0    448 KBytes       
[  5]   1.00-2.00   sec  80.9 MBytes   679 Mbits/sec    0    469 KBytes       
[  5]   2.00-3.00   sec  74.1 MBytes   622 Mbits/sec    0    469 KBytes       
[  5]   3.00-4.00   sec  83.5 MBytes   700 Mbits/sec    0    469 KBytes       
[  5]   4.00-5.00   sec  85.1 MBytes   714 Mbits/sec    0    469 KBytes       
[  5]   5.00-6.00   sec  85.3 MBytes   716 Mbits/sec    0    469 KBytes       
[  5]   6.00-7.00   sec  70.3 MBytes   590 Mbits/sec  241    356 KBytes       
[  5]   7.00-8.00   sec  84.1 MBytes   706 Mbits/sec    0    409 KBytes       
[  5]   8.00-9.00   sec  86.7 MBytes   728 Mbits/sec    0    434 KBytes       
[  5]   9.00-10.00  sec  86.0 MBytes   721 Mbits/sec    0    434 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   819 MBytes   687 Mbits/sec  241             sender
[  5]   0.00-10.00  sec   818 MBytes   686 Mbits/sec                  receiver

[werwolf@work] ~  
❯ iperf3 -c ZEROTIERTUNNELIP
Connecting to host 192.168.201.254, port 5201
[  5] local 192.168.201.8 port 54590 connected to 192.168.201.254 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   733 KBytes  6.00 Mbits/sec    0    172 KBytes       
[  5]   1.00-2.00   sec   802 KBytes  6.57 Mbits/sec    2    137 KBytes       
[  5]   2.00-3.00   sec   494 KBytes  4.04 Mbits/sec    4    150 KBytes       
[  5]   3.00-4.00   sec   309 KBytes  2.53 Mbits/sec   31   85.9 KBytes       
[  5]   4.00-5.00   sec   556 KBytes  4.55 Mbits/sec    0   99.3 KBytes       
[  5]   5.00-6.00   sec   247 KBytes  2.02 Mbits/sec    6   10.7 KBytes       
[  5]   6.00-7.00   sec   247 KBytes  2.02 Mbits/sec   30   83.2 KBytes       
[  5]   7.00-8.00   sec   494 KBytes  4.05 Mbits/sec    0   99.3 KBytes       
[  5]   8.00-9.00   sec   247 KBytes  2.02 Mbits/sec    2   53.7 KBytes       
[  5]   9.00-10.00  sec   247 KBytes  2.02 Mbits/sec    0   61.7 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  4.27 MBytes  3.58 Mbits/sec   75             sender
[  5]   0.00-10.18  sec  3.89 MBytes  3.20 Mbits/sec                  receiver
ppoozl commented 2 years ago

Is this going to be fixed. This problem still exists.

glimberg commented 2 years ago

@ppoozl It's a warning, not an error. May be a little bit noisy in the logs but it's not a problem

ppoozl commented 2 years ago

Yes I know, but it flushes away normal kernel message. And It's not ideal for me to check kernel message for other app's message which could be critial.

gonzaleztroyano commented 1 year ago

Hello there!

Same problem here 🥲.

user@server:~# dmesg | tail
[1607614.780744] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[1607629.506936] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[1607644.824036] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[1607674.864541] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[1607694.598098] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[1607704.912010] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[1607734.966551] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[1607759.710666] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[1607765.010967] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[1607795.062031] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
user@server:~# zerotier-cli -v
1.10.1
user@server:~# uname -a
Linux HOSTNAME 4.15.0-192-generic #203-Ubuntu SMP Wed Aug 10 17:40:03 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Do not hesitate to contact me in case more details are needed.

NeoChen1024 commented 9 months ago

Same problem on Arch Linux with Kernel 6.6.1 and ZeroTier-One 1.12.2

[210782.031982] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[210812.036958] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[210813.086834] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[210842.039991] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[210872.042404] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[210878.120603] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[210902.045144] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[210932.048399] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[210943.148359] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[210962.050835] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[210992.052526] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[211008.166348] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[211022.053890] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[211052.056213] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.

Kernel: 6.6.1-zen1-1-zen #1 ZEN SMP PREEMPT_DYNAMIC Wed, 08 Nov 2023 16:05:16 +0000 x86_64 GNU/Linux

Eyre-S commented 4 months ago

I am using latest Arch Linux for my HomeServe and it consistently output message to the TTY. It makes TTY very hard to use.

[ 1573.038934] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[ 1593.486149] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[ 1603.071634] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[ 1633.105198] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[ 1658.531461] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[ 1663.135385] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[ 1693.169760] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[ 1723.199621] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[ 1723.630905] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
[ 1753.228279] netlink: 8 bytes leftover after parsing attributes in process `zerotier-one'.
$ zerotier-cli -v
1.14.0

$ cat /proc/version
Linux version 6.6.30-1-lts (linux-lts@archlinux) (gcc (GCC) 13.2.1 20240417, GNU ld (GNU Binutils) 2.42.0) #1 SMP
PREEMPT_DYNAMIC Thu, 02 May 2024 18:20:28 +0000