outputPort is always 0 in flow samples

[mayuresh82]

im testing hsflowd version 2.0.11 on Cumulus Linux 4.2.1 and in my simple setup receiver ---(index3) switch (index5)----- sender/collector

the flow samples are always being sent with outputPort 0 as evident from the sflowtool dump:

startSample ----------------------
sampleType_tag 0:1
sampleType FLOWSAMPLE
sampleSequenceNo 3093
sourceId 0:5
meanSkipCount 1000
samplePool 3093000
dropEvents 0
inputPort 5
outputPort 0
flowBlock_tag 0:1
flowSampleType HEADER
headerProtocol 1
sampledPacketSize 1530
strippedBytes 4
headerLen 128
headerBytes 08-00-27-71-E3-C0-08-00-27-3D-B7-CA-08-00-45-00-05-DA-E9-BC-40-00-40-11-3C-F1-0A-01-03-64-01-00-00-01-E6-8A-13-89-05-C6-54-0D-00-01-20-0D-5F-F9-64-A4-00-00-72-1A-32-33-34-35-00-00-00-00-30-31-32-33-34-35-36-37-38-39-30-31-32-33-34-35-36-37-38-39-30-31-32-33-34-35-36-37-38-39-30-31-32-33-34-35-36-37-38-39-30-31-32-33-34-35-36-37-38-39-30-31-32-33-34-35-36-37-38-39-30-31-32-33-34-35
dstMAC 08002771e3c0
srcMAC 0800273db7ca
IPSize 1512
ip.tot_len 1498
srcIP 10.1.3.100
dstIP 1.0.0.1
IPProtocol 17
IPTOS 0
IPTTL 64
IPID 48361
UDPSrcPort 59018
UDPDstPort 5001
UDPBytes 1478
endSample   ----------------------
endDatagram   =================================

Please let me know if you need more information ?

[toreanderson]

I could be wrong, but I suspect the forwarding plane of the switch samples packets in the ingress pipeline, i.e., before the routing lookup is done and thus before the output interface is known.

[mayuresh82]

I am using psample which should be kernel based and not tied to switch subsystem. I even tried pcap sampling with { dev = inputIntf } and dev = { outputIntf } but still no outputPort info. Is there any way to enable egress sampling ?

[mayuresh82]

debug output :

takeSample: hook=0 tap=swp3 in=swp3 out=<not found> pkt_len=1512 cap_len=1498 mac_len=14 (0800273DB7CA -> 08002771E3C0 et=0x0800)
dbg2: selected sampler swp3 ifIndex=5
dbg1: psample netlink (type=24) CMD = 0
dbg3: psample: grp=1
dbg2: psample: in=5 out=0 n=1000 seq=84 drops=0 pktlen=1512
111

[mayuresh82]

For posterity : I Was able to get egress sampling working using NFLOG and POSTROUTING:

sudo iptables -t mangle -I POSTROUTING statistic mode random probability 0.001 nflog-prefix  SFLOW nflog-group 5

hsflowd.conf:

 nflog { group=5 probability=0.001 }

debug:

takeSample: hook=4 tap=<no tap> in=<not found> out=swp1 pkt_len=1498 cap_len=1498 mac_len=0 ( ->  et=0x0000)
dbg2: selected sampler swp1 ifIndex=3
netlink (1572 bytes left) msg [len=1552 type=1024 flags=0x0 seq=0 pid=0]

[toreanderson]

For what it is worth, my Edge-Core AS7326s (Trident3 ASIC) running CL4.2 with a completely minimal/default hsflowd.conf – essentially just sflow { collector { ip=x udpport=y } } – do include the output interface in the exported sFlow frames.

[mayuresh82]

@toreanderson i was running Cumulus VX (VM based) on Virtualbox - it may have something to do with that setup in my case.

[sflow]

In hardware switches with ECMP load-balancing it is often only the ASIC who knows the real physical egress port for a forwarded packet, so it's important for the ASIC to pass that information up with the packet-sample, and some do.

At some point we may extend hsflowd to probe the routing tables via NetLink. That would be to populate the extended_router structure, but if the lookup indicates just one possible egress port then it could potentially fill that in as well. Perhaps that would address this particular Cumulus VX case(?), but probably not if the routing table offered several ECMP alternatives. So to get the full picture in a software router we might even have to instrument or simulate the load-balancing hash function.

In many cases the hard-to-know egress becomes the easy-to-know ingress at the next step along in the packet path, but there are definitely some analysis scenarios where knowing the actual egress port is pivotal. It is a high-value measurement.