search

bitkeks / python-netflow-v9-softflowd

PyPI "netflow" package. NetFlow v9 parser, collector and analyzer implemented in Python 3. Developed and tested with softflowd
https://bitkeks.eu/blog/2016/08/collecting-netflow-v9-on-openwrt.html
MIT License
116 stars 59 forks source link

IPFIX #20

Closed schadom closed 4 years ago

schadom commented 4 years ago

Thanks for this great project. Any plans for supporting IPFIX?

bitkeks commented 4 years ago

Hi, thanks for the nice feedback! I had planned to include it some time, but am working on stability right now. The library was refactored the last two days and is now an official package on PyPI: https://pypi.org/project/netflow/ You're very welcome to experiment with IPFIX of course. Feel free to add a netflow.ipfix module!

bitkeks commented 4 years ago

@schadom regarding your mentioned issue in manuelkasper/AS-Stats#88, you said

Currently I'm working on ip2as-lookups

I recently built a tool which mapped hostname -> IPs -> ASN. Maybe you find something useful in the repo: https://github.com/bitkeks/fediverse-infra-analysis/blob/master/ip2asn.py (links to my related blog article in the README).

schadom commented 4 years ago

@bitkeks world is small it seems :-) I did the same already, a Python script which fetches the pfx2as data daily from http://data.caida.org/datasets/routing and then uses this to do IP2AS lookups. This is more accurate than relying on - for example ripe-delegated list or other sources - because this really has fresh prefix/origin-as data. as-name is then resolved via whois.cymru.com by looking up the AS number.

This is already built-in into as-stats-python which is using python-netflow-v9-softflowd. So we need IPFIX support next :-)

bitkeks commented 4 years ago

That's good to hear! Currently investigating the diff for v9 to IPFIX. I'd also try to adapt the collector to speak TCP. And SCTP, but that might be more of a case of learning purpose..

Maybe this also helps resolving #21

schadom commented 4 years ago

Another alternative would be to do IP2AS and AS-NAME lookups by querying whois.cymru.com (https://asn.cymru.com) and store the results into a file. The downside of both methods is, that this won't work on installations without internet access.

is now an official package on PyPI: https://pypi.org/project/netflow/

thanks for the hint!

I will push the repo to github later on.

bitkeks commented 4 years ago

IPFIX is now included in the package and available to the collector. Could anyone of you test it in your setups? I'd then release a new version on PyPI with IPFIX. Thanks!

schadom commented 4 years ago

@bitkeks wow, that's amazing! thank you very much! will test later 👍

bitkeks commented 4 years ago

After adding some more tests, netflow v0.10.1 with IPFIX support is now published on PyPI: https://pypi.org/project/netflow/0.10.1/