Enhancement idea: configuration option to write packets in pcapng format

[dmiller-nmap]

Npcap inherits WinPcap's interface, which means packets are written in something resembling the "pcap" file format (https://datatracker.ietf.org/doc/html/draft-gharris-opsawg-pcap). If we supported writing the extensible pcapng format (https://datatracker.ietf.org/doc/html/draft-tuexen-opsawg-pcapng), we could do things like:

• Report statistics directly in the packet stream, including adding new statistics as we like without being concerned about backwards compatibility.
• Avoid clearing the entire packet buffer when a capture configuration change is made, preferring instead to inject a block of appropriately descriptive metadata indicating exactly when the change took place (e.g. a new packet filter, etc.)
• Report diagnostic info directly in the packet stream, such as changes to the adapter reported via Status Indications or average buffer occupancy.

[guyharris]

Please file this as a libpcap issue as it applies to all OSes, not just Windows (I have a few Linux goodies that can be done with pcapng, for example).

That way, discussion can happen here, and include libpcap users other than Npcap users.

It shouldn't be a configuration option, it should be an API option; for one thing, the API for writing pcapng would be more complicated than the API for writing pcap, and the change should also include full support for reading pcapng (as opposed to the "silently open and read them as best we can given the pcap-oriented capture reading API) and APIs for capturing in "pcapng mode" (e.g., supplying IDBs for the interfaces on which the capturing is being done).

[guyharris]

Report statistics directly in the packet stream, including adding new statistics as we like without being concerned about backwards compatibility.

Yes - pcap_stats() is really feeble. You can't add new statistics and it can't indicate that some statistics aren't supported, it just has to return 0. Something more pcapng-oriented, e.g. where you provide a list of statistics options that you want, and something like an ISB, but containing only the options that can be supported, are provided.

[guyharris]

Avoid clearing the entire packet buffer when a capture configuration change is made, preferring instead to inject a block of appropriately descriptive metadata indicating exactly when the change took place (e.g. a new packet filter, etc.)

That would require adding an "Interface Update Block" or something such as that. You can specify the initial filter in an IDB, but you can't say "the filter changed". (It also requires assistance from the capture mechanism code, so that it indicates at what point in the packets being fed to userland the change occurred.)

[guyharris]

Report diagnostic info directly in the packet stream, such as changes to the adapter reported via Status Indications

That would require adding blocks to provide network-layer indications, which might be a Good Idea (and could possibly be done on other platforms.)

or average buffer occupancy.

That sounds like a statistic - a candidate for a new option for the ISB?

[guyharris]

(And, yes, a pcapng-capable capture/read/write API is something I've been working on.)

[dmiller-nmap]

@guyharris I'm glad to hear that you're working on this for libpcap. Npcap will of course continue using libpcap as its API, so I did not intend to suggest that we would extend the libpcap API to present pcapng data to the user. Rather, in anticipation of such an addition to libpcap, I was suggesting that Npcap could extend the Packet.dll API to allow libpcap to receive pcapng data directly from the driver. The particular applications I listed would then be possible to add in an extensible manner without breaking backwards compatibility with older programs. I think we'd probably use an additional flag for PacketSetMode() to activate this.