PF_RING DNA support

[waszi]

I was trying to run fastnetmon with PF_RING DNA based drivers but there is some issue with packet parsing. Packets are counted but it could not detect IPs.

[pavel-odintsov]

Hello, waszi!

Please try to change this line:

bool we_use_pf_ring_in_kernel_parser = false;

And recompile project. In DNA/ZC mode packet parser completely disabled and we should use parser from library.

If of possible please share your hardware, network load and cpu load (from htop) before and after enabling DNA.

[waszi]

I have recompiled app with this change. I've attached screenshot from results but they are weird. I am using Intel 82599 NIC with DNA driver on Supermicro platform (Xeon X3440). tcpdump from pf_ring is able to parse all packets without any problems so I think that DNA/kernel configuration is valid.

[pavel-odintsov]

This issue looks like bug in pfring_parse_pkt.

Please pull new code from Git and uncomment lines at end of parse_packet_pf_ring function:

char buffer[512];
pfring_print_parsed_pkt(buffer, 512, p, h);
logger<<log4cpp::Priority::INFO<<buffer;

After this please collect data for few minutes and upload log /var/log/fastnetmon.log anywhere.

[waszi]

http://waszi.xshells.net/fastnetmon.log.gz

[pavel-odintsov]

Thank you!

Did you test FastNetMon in normal non - DNA/ non - ZC mode? Is it works correctly on one-copy mode?

[waszi]

In non zc-mode/DNA it is working fine. DNA mode with tcpdump from pf_ring package is also working fine.

[pavel-odintsov]

Thank you for your report. I reproduced this issue locally. I also created proof of concept https://github.com/FastVPSEestiOu/fastnetmon/blob/master/pfring_parser_zc_issue.c and reported this bug to PF_RING maillist.

My local weird is looking very similar to your:

[90:E2:BA:49:85:C8 -> 5C:5E:AB:24:0F:C0] [IPv4][159.253.18.207:0 -> 95.159.190.225:0] [l3_proto=TCP][hash=2845356851][tos=0][tcp_seq_num=0] [caplen=128][len=1514][parsed_header_len=0][eth_offset=0][l3_offset=14][l4_offset=34][payload_offset=0]
[90:E2:BA:49:85:C8 -> 5C:5E:AB:24:0F:C0] [IPv4][159.253.18.207:0 -> 95.159.190.225:0] [l3_proto=TCP][hash=4262221779][tos=0][tcp_seq_num=0] [caplen=66][len=66][parsed_header_len=0][eth_offset=0][l3_offset=14][l4_offset=34][payload_offset=0]
[90:E2:BA:49:85:C8 -> 5C:5E:AB:24:0F:C0] [IPv4][159.253.18.207:0 -> 95.159.190.225:0] [l3_proto=TCP][hash=2559938100][tos=0][tcp_seq_num=0] [caplen=66][len=66][parsed_header_len=0][eth_offset=0][l3_offset=14][l4_offset=34][payload_offset=0]
[90:E2:BA:49:85:C8 -> 5C:5E:AB:24:0F:C0] [IPv4][159.253.18.207:0 -> 95.159.190.225:0] [l3_proto=TCP][hash=2559938100][tos=0][tcp_seq_num=0] [caplen=66][len=66][parsed_header_len=0][eth_offset=0][l3_offset=14][l4_offset=34][payload_offset=0]
[90:E2:BA:49:85:C8 -> 5C:5E:AB:24:0F:C0] [IPv4][159.253.18.207:0 -> 95.159.190.225:0] [l3_proto=TCP][hash=2845356851][tos=0][tcp_seq_num=0] [caplen=128][len=1514][parsed_header_len=0][eth_offset=0][l3_offset=14][l4_offset=34][payload_offset=0]
[90:E2:BA:49:85:C8 -> 5C:5E:AB:24:0F:C0] [IPv4][159.253.18.207:0 -> 95.159.190.225:0] [l3_proto=TCP][hash=2802630656][tos=0][tcp_seq_num=0] [caplen=66][len=66][parsed_header_len=0][eth_offset=0][l3_offset=14][l4_offset=34][payload_offset=0]

P.S. you can remove http://waszi.xshells.net/fastnetmon.log.gz :)

[pavel-odintsov]

Hello, waszi!

Please try current version from repo with option bool we_use_pf_ring_in_kernel_parser = false;

[pavel-odintsov]

Since https://github.com/FastVPSEestiOu/fastnetmon/commit/3f742b705fd3b61cc4b1184dd86f3b5185f7c03e you can specify prefix zc for interfaces in config: zc:eth3 without any code changes :)

[waszi]

I would be very happy if you would add checking for dnaX devices :) Current version is working fine with DNA devices. But still I have been unable to get zero packet loss with more than 2.5Mpps (15% pkts lost). In the next week I will try this again with better CPU (X3440 is pretty old, I wonder how it will work with Xeon E3 or E5)

[pavel-odintsov]

Hello! Excellent news! Could you provide screenshot of htop and top from your machine while packetloss occur?

[pavel-odintsov]

For temporary work around you can try version from repo and disable connection tracking in config file:

 enable_connection_tracking = off

It will disable connection tracking and got 2-3 times cpu load reduction.

[pavel-odintsov]

And finally. Please pull current code from repo and do following (with addition to enable_connection_tracking = off):

bool enable_pfring_multi_channel_mode = true;

After this fix load_driver.sh and set number of NIC queues == number of logical cpus:

insmod ./ixgbe.ko MQ=1,1,1,1 RSS=4,4,4,4

After this, FastNetMon will load all cores equally. Any future optimizations possible only with compete switching to ZC API and need some time.

[vps2fast]

I added compete ZC API for PF_RING with balancer and lightweight packet parser. Please set pf_ring_zc_api_mode = true and recompile project.

I can handle about 5.1 MPPS now on 4 core E5-2407 0 @ 2.20GHz:

  1  [|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||100.0%]     Tasks: 25, 11 thr, 47 kthr; 2 running
  2  [|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||100.0%]     Load average: 2.59 1.73 1.33 
  3  [|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||100.0%]     Uptime: 02:43:59
  4  [|||||||||||||||||||||||||||||||||||||||||||||||                                         49.7%]

IPs ordered by: packets (use keys 'b'/'p'/'f' for change) and use 'q' for quit
Incoming traffic        5147067 pps   2356 mbps      0 flows

[pavel-odintsov]

ZC support can be enabled now with following flags in config file without any recompilation:

interfaces = zc:eth4
enable_pf_ring_zc_mode = on

If you have any questions you can email me directly pavel.odintsov@gmail.com. This ticket I will close.