Can not generate BPF extensions on dead handle.

[shemminger]

Is there a way to generate BPF code to use extensions around vlan's when compiling on a non-live pcap handle?

I am adding support for pcap filters to DPDK. The code works by doing:

pcap = pcap_open_dead(DLT_EN10MB, snaplen);
pcap_compile(pcap, &bf, filter_str,
         1, PCAP_NETMASK_UNKNOWN);

And then the resulting BPF code is converted to eBPF interpreted by DPDK.

This works for most cases, but the DPDK has options to strip VLAN's like Linux kernel. The converter code could handle that BUT the pcap_compile won't generate required BPF extensions on a handle created in this way.

Don't want to create a live handle because don't want to be capturingpackets in the kernel environment.

Maybe an extension like: pcap_compile_with_bpf_extensions(...)

[mcr]

Yes, that sounds reasonable, but we need to avoid more _with_foo_bar() stuff, and do something that is more easily turned into an ABI. The short is that there isn't a way.

[guyharris]

I am adding support for pcap filters to DPDK.

Do you mean that pcap filters don't work when you capture on a DPDK device using libpcap (i.e., opening a DPDK device, and capturing on it, using libpcap APIs), and you're fixing that, or do you mean you're adding support for pcap filter expressions when not using libpcap to do DPDK captures?

If it's the latter, then that seems to indicate that what's needed is a separate library to compile pcap filters into various machine languages.

Don't want to create a live handle because don't want to be capturingpackets in the kernel environment.

If you capture on a DPDK device using libpcap, you're not, as far as I know, capturing in the kernel environment.

[shemminger]

To be more clear about what this is trying to do:

• The network processing application (for example OpenVswitch) is a DPDK primary process which interacts with usermode network drivers.
• Application in DPDK (dpdk-dumpcap) runs as a secondary process (communicating over IPC/shared memory to primary):
• It compiles a filter using pcap_open_dead/pcap_compile and gets BPF result.
• It passes the BPF program off to function rte_bpf_convert() in DPDK that converts BPF to eBPF
• Finally the application then communicates the request then makes a request to the primary process
• Resulting eBPF is handled by DPDK as part of the capture process.

The key point is at the compile step. Since the application is using pcap_open_dead, the code generation (gencode.c) will not generate BPF extensions to check for VLAN.

[guyharris]

The network processing application (for example OpenVswitch) is a DPDK primary process which interacts with usermode network drivers.

So it's not doing so by using libpcap to open a DPDK device?

[shemminger]

Correct. It is not using libpcap directly on DPDK device. Libpcap has a DPDK shim but it only works when it is the primary process, making it useless except for when libpcap is being used for some SPAN type application where all packets are going to the pcap application only.

It is on my future wish list to rewrite/revise/replace the current libpcap dpdk stuff with something more useful and integrated with upstream, but that is a larger project.

[guyharris]

OK, so perhaps what's needed here is a separate API - and perhaps a separate library, carving the filter-to-code routines out into a separate library, for use by libpcap and other software that wants to generate filter code from a pcap filter expression.

For one thing, that'd get rid of the pcap_open_dead() hack; it would also allow for multiple flavors of generated code, including, but not necessarily limited to, "needs special tweaks for VLANs".

[shemminger]

Sure, having gencode standalone would be good.

[mcr]

If the goal is to have applications (such as tcpdump or wireshark) be able to use libpcap to capture from DPDK, then it sounds like you really ought to have a backend driver next to pcap-linux.c that talks to DPDK. Then you'd pcap_open() with an appropriate argument to say DPDK, or you'd use a libpcap-dpdk.so, or something like that.

[mcr]

Sure, having gencode standalone would be good.

Maybe you want to send patches to build a libpcapcode that is appropriate for your use. This would be a new .so that libpcap would ship.

[guyharris]

If the goal is to have applications (such as tcpdump or wireshark) be able to use libpcap to capture from DPDK, then it sounds like you really ought to have a backend driver next to pcap-linux.c that talks to DPDK.

A backend driver called something such as pcap-dpdk.c? :-)

[shemminger]

The pcap-dpdk.c driver wants to be the primary process and own the device, which makes it useless for most applications. See my comment above.

[mcr]

The pcap-dpdk.c driver wants to be the primary process and own the device, which makes it useless for most applications. See my comment above.

So, we are happy to take another version, changes to this version, obsoleting this code if it's in practice, useless.

[guyharris]

The pcap-dpdk.c driver wants to be the primary process and own the device, which makes it useless for most applications. See my comment above.

So, we are happy to take another version, changes to this version, obsoleting this code if it's in practice, useless.

"Most applications" != "all applications". pcap-dpdk.c was introduced in pull request #790, with the description

1. Pcap-dpdk provides libpcap the ability to use DPDK with the device name as dpdk:[portid], such as dpdk:0.
2. DPDK is a set of libraries and drivers for fast packet processing. (https://www.dpdk.org/)

so presumably the intent was to support using DPDK as a mechanism for doing fast packet processing when capturing network traffic, i.e. packet capture is the application for which pcap-dpdk.c is intended.

It sounds as if the goal here is to allow some other application for DPDK to log the packets that it processed. This is somewhat like having a packet-logging mechanism in some networking hardware, where the purpose of the hardware is something other than packet capture, such as routing or switching packets, and packet capture is a diagnostic tool that it offers.

[shemminger]

My longer term plan is to support all flavors.

• DPDK only using the rte_pdump library and application that is part of DPDK
• Libpcap used in primary process (current pcap-dpdk.c)

• Libpcap used from secondary process (new support using ring buffer from DPDK pdump library).

  The DPDK has support for primary and secondary process an application can pass "auto" and run either way. So if you ran tcpdump/wireshark while no other DPDK process is running, you get the code from PR #790. If you ran those while some other application (OVS, VPP, Cisco, Juniper,....) is using DPDK, then it would work like the native Linux network stack does.

First step is to get the native DPDK implementation optimized and the infrastructure in place for this releas.

[guyharris]

DPDK only using the rte_pdump library and application that is part of DPDK

From what I could find, that's "dumping to a file(?) packets passing through some other DPDK application".

Libpcap used in primary process (current pcap-dpdk.c)

That's "fast packet capture using the libpcap APIs (so that applications using libpcap, when running with a libpcap that supports DPDK, can capture using DPDK rather than the regular kernel networking and packet capture data path".

Libpcap used from secondary process (new support using ring buffer from DPDK pdump library).

So is that "capturing packets passing through some other DPDK application, using the libpcap APIs (so that applications using libpcap, when running with an application with a libpcap that supports getting packets from some other DPDK application, can capture those packets)"?

Where is there good documentation that covers all of pdump? Does "DPDK pdump Library and pdump Tool" suffice for that?