search

sflow / host-sflow

host-sflow agent
http://sflow.net
Other
152 stars 55 forks source link

hsflowd fails to identify pppoe0 interface #68

Open frebib opened 1 week ago

frebib commented 1 week ago

ref https://vyos.dev/T6762

Running hsflowd with -ddd also tells me the same thing:

mod_pcap:macsrc=1E43BED99C90, macdst=FEAA9ADDF555
mod_pcap:srcdev=bond0.40(10)(peer=0)
takeSample: hook=0 tap=eth2 in=bond0.40 out=<not found> pkt_len=76 cap_len=76 mac_len=14 (1E43BED99C90 -> FEAA9ADDF555 et=0x8100)
dbg2:selected sampler eth2 ifIndex=4
mod_pcap:macsrc=FEAA9ADDF555, macdst=1E43BED99C90
mod_pcap:dstdev=bond0.40(10)(peer=0)
takeSample: hook=0 tap=eth2 in=<not found> out=bond0.40 pkt_len=163 cap_len=114 mac_len=14 (FEAA9ADDF555 -> 1E43BED99C90 et=0x8100)

Looks like Linux doesn't provide the PPPoE interface MAC via SIOCGIFHWADDR, which I think makes sense

ioctl(7, SIOCGIFFLAGS, {ifr_name="pppoe0", ifr_flags=IFF_UP|IFF_POINTOPOINT|IFF_RUNNING|IFF_NOARP|IFF_MULTICAST}) = 0
ioctl(7, SIOCGIFHWADDR, {ifr_name="pppoe0", ifr_hwaddr={sa_family=ARPHRD_PPP, sa_data=00:00:00:00:00:00:00:00:00:00:00:00:00:00}}) = 0
ioctl(7, SIOCGIFINDEX, {ifr_name="pppoe0", ifr_ifindex=13}) = 0
ioctl(7, SIOCGIFADDR, {ifr_name="pppoe0", ifr_addr={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("??.??.??.??")}}) = 0
ioctl(7, SIOCETHTOOL, 0x7ffedafde390)   = 0
ioctl(7, SIOCETHTOOL, 0x7ffedafde390)   = -1 EOPNOTSUPP (Operation not supported)
dbg1:ETHTOOL_GMODULEINF0 pppoe0 failed : Operation not supported
ioctl(7, SIOCETHTOOL, 0x7ffedafde390)   = -1 EOPNOTSUPP (Operation not supported)
ioctl(7, SIOCETHTOOL, 0x7ffedafde390)   = -1 EOPNOTSUPP (Operation not supported)
ioctl(7, SIOCETHTOOL, 0x7ffedafde390)   = 0

Same as

frebib@zeus:~$ cat /sys/class/net/pppoe0/address

frebib@zeus:~$

iproute2 also shows this too (where bond0.10 is the underlying interface)

7: bond0.10@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9710 qdisc cake state UP mode DEFAULT group default qlen 1000
    link/ether 1e:43:be:d9:9c:90 brd ff:ff:ff:ff:ff:ff
13: pppoe0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1492 qdisc fq_codel state UNKNOWN group default qlen 3
    link/ppp
    inet <snip> peer <snip>/32 scope global pppoe0
       valid_lft forever preferred_lft forever
    inet6 <snip>::1/64 scope global dynamic mngtmpaddr proto kernel_ra
       valid_lft 2591533sec preferred_lft 604333sec
    inet6 fe80::f551:75ff:fedc:f28d/64 scope link
       valid_lft forever preferred_lft forever
    inet6 fe80::1 peer fe80::827f:f8ff:fe74:c8db/128 scope link
       valid_lft forever preferred_lft forever

Here is a pcap of a ping to a resolver showing the PPP packets in either direction

21:41:33.485179 1e:43:be:d9:9c:90 > 80:7f:f8:74:c8:db, ethertype 802.1Q (0x8100), length 110: vlan 10, p 0, ethertype PPPoE S (0x8864), PPPoE  [ses 0x72d] IP (0x0021), length 86: ??.??.??.?? > 1.1.1.1: ICMP echo request, id 1, seq 1, length 64
21:41:33.487988 80:7f:f8:74:c8:db > 1e:43:be:d9:9c:90, ethertype 802.1Q (0x8100), length 110: vlan 10, p 0, ethertype PPPoE S (0x8864), PPPoE  [ses 0x72d] IP (0x0021), length 86: 1.1.1.1 > ??.??.??.??: ICMP echo reply, id 1, seq 1, length 64

The only place I see that MAC address is in proc/net/pppoe

frebib@zeus:~$ cat /proc/net/pppoe
Id       Address              Device
00002D07 80:7f:f8:74:c8:db bond0.10

I presume it's synthesised by the kernel for the virtual PPP device

sflow commented 1 week ago

To understand this better I have a few questions:

Q1: is there a "bond0.40" netdev on this server that also has MAC address == 1E43BED99C90, same as bond0.10 ? Q2: What is the MAC address associated with the physical interface eth2? Q3: the pcap you took of the ping - where did you tap to get those packets? In other words, what was the tcpdump command that you used? Was it something like "tcpdump -i eth2 ..." Q4: please send the contents of /etc/hsflowd.conf Q5: an example of what a packet samples looks like in sflowtool(1) would be helpful too.

From the point of view of the sFlow standard, the correct thing to export as the inputPort/outputPort is probably the ifIndex of the physical interface "eth2". The collector would still see the 802.1Q header to get the VLAN tag and the PPPoE header to get the PPP port number. But it looks like hsflowd is trying to follow the MAC that it finds in the sampled packet header to arrive at some deeper truth, when maybe it's shouldn't in this case.

frebib commented 1 week ago
  1. Yes. They're both parented to the same bond
  2. 4: eth2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9710 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000
link/ether 1e:43:be:d9:9c:90 brd ff:ff:ff:ff:ff:ff permaddr f8:f2:1e:3a:67:59
altname enp1s0f1
6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9710 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 1e:43:be:d9:9c:90 brd ff:ff:ff:ff:ff:ff
7: bond0.10@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9710 qdisc cake state UP mode DEFAULT group default qlen 1000
link/ether 1e:43:be:d9:9c:90 brd ff:ff:ff:ff:ff:ff
8: bond0.20@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9710 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 1e:43:be:d9:9c:90 brd ff:ff:ff:ff:ff:ff
9: bond0.30@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9710 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 1e:43:be:d9:9c:90 brd ff:ff:ff:ff:ff:ff
10: bond0.40@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9710 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 1e:43:be:d9:9c:90 brd ff:ff:ff:ff:ff:ff
13: pppoe0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1492 qdisc fq_codel state UNKNOWN mode DEFAULT group default qlen 3
link/ppp
alias wan
  3. eth2 specifically: tcpdump -enni eth2 ether[0x0c:2] == 0x8863 or ether[0x0c:2] == 0x8864 (to show PPP ethertype packets). Using -i any/LINUX_SLL2 doesn't show nearly as much PPP which I understand is "by design"/a thing that Linux does, apparently.
  4. 
frebib@zeus# cat /run/sflow/hsflowd.conf
# Genereated by /usr/libexec/vyos/conf_mode/system_sflow.py
# Parameters http://sflow.net/host-sflow-linux-config.php

sflow { polling=30 sampling=1000 sampling.bps_ratio=0 agentIP= agent=lo collector { ip = udpport = 6343 } pcap { dev=eth2 } dbus { } }


5. I'll have to get back to you on this another day, I've run out of time for today.

Thanks for the questions so far :)
sflow commented 1 week ago

OK, that all makes sense so far except that it's not clear to me exactly when the MAC address is changed from 1e:43:be:d9:9c:90 to 80:7f:f8:74:c8:db. I would have guessed that when tcpdump uses a libpcap call to tap eth2 it would look the same as when hsflowd mod_pcap uses the a libpcap call to tap eth2. However it looks like they are tapping at different places in the pipeline.

Looking forward to seeing the packet-sample as printed by sflowtool. Perhaps that will make things more clear. When you send that, please also reiterate, for this issue thread, what you consider to be wrong there.