rickyzhang82
commented
5 years ago In fact , why not convert the bespoke build script to traditional autoconf with makefile or even CMake?
Closed rickyzhang82 closed 5 years ago
rickyzhang82
commented
5 years ago In fact , why not convert the bespoke build script to traditional autoconf with makefile or even CMake?
seladb
commented
5 years ago I don't remember the exact reason why libstdc++ static was needed, but I do remember it was needed to build PcapPlusPlus successfully. You're can of course try to remove this dependency and see if PcapPlusPlus builds and runs correctly. Please let me know if you succeed. Actually it'd be great to add official FreeBSD support to PcapPlusPlus so based on your work we can decide the best path forward.
Regarding your question about autoconf/CMake - please see https://github.com/seladb/PcapPlusPlus/issues/164. This is work-in-progress, but it's going on pretty slowly and I can't say when it'll be ready. The challenging part is proper support of the different OS's and frameworks currently being supported in PcapPlusPlus
rickyzhang82
commented
5 years ago I skim through the code. The structure is well written. I'm not familiar with all dependencies like PF_RING or DPDK. It should be extremely easy to write a simple CMake to handle libpcap dependency in all *nix like OS.
I will give a try in Linux first and see if it fits my research project. Once it is done, I will try it on FreeBSD.
PS: I love your documentation. Job well done and hats off to you.
seladb
commented
5 years ago Thank you so much @rickyzhang82 for the kind words, I really appreciate that!
PF_RING and DPDK are frameworks for high rate packet processing and as far as I know are supported only on Linux. PcapPlusPlus provides a C++ wrapper for them. Supporting them on FreeBSD is probably not needed.
As a first step for supporting FreeBSD I'd recommend using the Linux configuration script configure-linux.sh and see what tweaks and changes are needed there. Eventually we'll probably create a new config script for FreeBSD (maybe configure-freebsd.sh) which will include the dependencies and parameters needed for FreeBSD. Another item we'll need to think about is how to add CI for FreeBSD as both TravisCI and Appveyor are currently not supporting it
rickyzhang82
commented
5 years ago Sorry, I don't want to hijack the conversation in FreeBSD. But I want to discuss my intention of using PcapPlusPlus with you. You have better insight than me. Please let me know if I should do it in another channel rather than here.
I'm doing a research project with my spare time to verify a patent filed by an evil regime, who claimed to employ the deep packet inspect technique to blockade free flow of information. In a nutshell, they collect network traffic, use first 16th packets in each TCP connection (both directions) as training data, train/inference a recurrent neutral network to classify target encrypted protocol over TCP.
To replicate the claim, I need to build a training data set in my home router. I used Netgate pfsense router at home. But to beef up storage and computing power, I installed a separate pfsense in my home server which has better CPU processing power and SSD storage.
I did a gap analysis on your example program TcpReassembly and library class TcpReassembly with my requirement. Here is the list of TODOs I think I should close the gap:
Bullet point 1 & 3 are easy to do. But bullet point 2 needs some hack in class TcpReassembly from PcapPlusPlus library. It may require to change the signature of 3 callback function OnTcpMessageReady, OnTcpMessageReady and OnTcpMessageReady.
What would you recommend me to do in terms of class design if I want to generalize these features and send a PR?
rickyzhang82
commented
5 years ago I sent your a PR #261 and verified it works in my pfsense router.
Travis doesn't support FreeBSD. The best way to keep your stuff tested is to make your repository as one of FreeBSD port. I'm not an expert in legal. FreeBSD is quite picky on license. If your license is compatible with FreeBSD license, I will write a port file for you as well.
seladb
commented
5 years ago Thanks for you PR! I'll review it shortly. Adding PcapPlusPlus to FreeBSD ports is a great idea but unfortunately I have no experience with this so I'd need your help.
Regarding CI - it's very important to have CI for FreeBSD to make sure future code changes don't break FreeBSD compatibility. TravisCI doesn't support FreeBSD but fortunately I found a CI that does: Cirrus CI. I've started playing with it for other purposes (improving PcapPlusPlus packaging and deployment) and I think we can use it just for FreeBSD. Would you like to give it a try?
seladb
commented
5 years ago Regarding the changes you propose in TcpReassembly (specifically bullet point 2) - I'm not sure why you need to keep the raw packet from the IP layer. If you need IP address information you can get if from TcpReassembly itself...
btw, maybe we can open a new GitHub issue for this or discuss this in PcapPlusPlus Google Group
rickyzhang82
commented
5 years ago I will look into Cirrus CI later tonight. It should be a piece of cake to just make it built. In fact, if we can package this as a FreeBSD port, more FreeBSD folks can use it out-of-the-box. In the end, I can just grab it from pfSense directly without my compiling.
I can't get your 3rd party memory leak checking works in FreeBSD. Therefore, I can't build your test program. If you want to get your test running in FreeBSD, you might need some help from others. But in FreeBSD the static libraries were built successfully. I verified the example app TcpReassembly works.
Let's discuss the bullet point 2 in Google group here.
seladb
commented
5 years ago The memory leak check is here if you want to take a look and suggest a fix: https://github.com/seladb/MemPlumber
I hope we can make it work on FreeBSD, the unit-tests are a very important part of the CI...
Also you can take a look at a very initial version of working with Cirrus CI: https://github.com/seladb/PcapPlusPlus-Deploy/blob/cirrus-ci/.cirrus.yml
rickyzhang82
commented
5 years ago I got the Cirrus CI done in my dev branch dev-pr-cirrus-ci:
https://github.com/rickyzhang82/PcapPlusPlus/tree/dev-pr-cirrus-ci
Check out the 3 commits and integrate it by yourself.
seladb
commented
5 years ago BIG THANK YOU for your significant contribution to this project!!! I really appreciate it!
Before I merge it to master I need to update the documentation and add FreeBSD support: https://pcapplusplus.github.io/
I'll do it soon and then merge your PR to master
rickyzhang82
commented
5 years ago Thanks for your great work! I found this repo through tcpdump/libpcap's related projects page. After some investigation, I decided to use yours to start my research project. Because I can tell from the clean code and the well-written doc that you must invest serious efforts to achieve this.
Regarding to sending FreeBSD port to upstream, it takes longer time than this PR. I will create a separate issue for this and come back when I'm done with my project.
seladb
commented
5 years ago Sure, thank you again for all your work!
BTW, I'm not sure about FreeBSD ports, but other package managers usually publish only official releases. In that case we'll need to create an official release. Please let me know
Hi,
I’m working on building PcapPlusPlus in pfsense, a derivative of FreeBSD.
Since there is no instructions for FreeBSD installation, I looked into configuration of Linux. It seems to have two dependencies: libpcap and libstdc++ static.
I found that libpcap is supported in FreeBSD port. But due to GPL license in libstdc++, I may have hard time to find it for FreeBSD 11.
I wonder why the requirements needs libstdc++ static in Linux? Is it required in FreeBSD?