mod_psample, mod_dropmon drops samples larger than 8124 bytes

[bluecmd]

Hi,

Currently mod_psample and mod_dropmon reads packet samples from netlink using a buffer size of HSP_PSAMPLE_READNL_RCV_BUF and HSP_DROPMON_READNL_RCV_BUF set to 8192. This is not enough to fit a packet with MTU 9000. Given extra metadata and such, the cut-off ends up being at >8124 bytes packets.

Currently the read from netlink is implemented like this:

static void readNetlink_PSAMPLE(EVMod *mod, EVSocket *sock, void *magic)
{
  HSP_mod_PSAMPLE *mdata = (HSP_mod_PSAMPLE *)mod->data;
  uint8_t recv_buf[HSP_PSAMPLE_READNL_RCV_BUF];
  int batch = 0;
  for( ; batch < HSP_PSAMPLE_READNL_BATCH; batch++) {
    int numbytes = recv(sock->fd, recv_buf, sizeof(recv_buf), 0);
    if(numbytes <= 0)
break;
    struct nlmsghdr *nlh = (struct nlmsghdr*) recv_buf;
    while(NLMSG_OK(nlh, numbytes)){
      // [..]
    }
  }
  // [..]
}

This causes the NLMSG_OK to reject the read as it is truncated and the event is dropped.

I suggest increasing these to buffers to 32 KiB.

[sflow]

Thanks for pointing this out. I just have one question: why do you suggest 32K? A UDP PDU can have up to 64K payload so perhaps something like 64K + 128 is necessary to cover (almost) all possibilities?

[bluecmd]

The largest MTU I have seen ASICs support have been 16 KIB, so that's why I figured 32 KiB "should be enough for everyone". However, that's just anecdotal. I tried finding a source for the largest MTU, and some people argued it is 2^32 which obviously isn't 1) realistic and 2) not suitable with the current way of reading from netlink.

I have nothing against your suggestion of increasing the size even more.

[bluecmd]

FWIW, I am now running with your proposed buffer size and can confirm I now see sFlow samples when running with >9000 bytes. Full patch @ https://github.com/kamelnetworks/sonic-buildimage/commit/f5d1689ea14d1597d47dc74dd28c549efbceaad6

[sflow]

Both the PSAMPLE and DROPMON channels allow for the packet to be truncated (e.g. just send the first 256 bytes and indicate the original size with PSAMPLE_ATTR_ORIGSIZE or NET_DM_ATTR_ORIG_LEN attributes respectively). So to run into this problem, did you first have to configure a large number for the desired header size? Or does your platform always send the whole sampled/dropped packet?

[bluecmd]

Well, not sure what to tell you other than that the platform I used, Teralynx based, did send the whole packet. I believe that the behavior you're describing is what Broadcom does, but our Teralynxes needed this patch.

[sflow]

I increased the buffer size to 16384 in commit 4f7abdc