private prefixes/ASNs

[acoul]

Greetings,

how easy would be to switch both databases to use an internal Prefix/ASN map file?

[acoul]

vtysh -c "sh ip bgp" | grep \/24 | cut -c 4-20,60-600 | sort | uniq | sed 's/ 0 //g' | sed 's/ / /g' | sed 's/ / /g' | grep " i" | sed 's/ i//g' | awk '{printf "16777216,16777471,\"AS"$NF" "$1"\"\n"}' > GeoIPASNum2.csv

vtysh -c "sh ip bgp" | grep \/24 | cut -c 4-20,60-600 | sort | uniq | sed 's/ 0 //g' | sed 's/ / /g' | sed 's/ / /g' | grep " i" | sed 's/ i//g' | awk '{printf $1"\t"$NF"\n"}' > ip_to_asn.db

[acoul]

this is a bit painful for a n00b. maybe this

[coxley]

Hey @acoul: Not sure how I didn't see this issue.

I do use pyasn in this lib. :) PyASN requires a local geomind dump iirc

[acoul]

So far my efforts to force using pyasn only have failed. creating a custom ip_to_asn.db and devnulling the GeoIPASNum2.csv still gets resolution of Internet ASNs against our private AS/Prefix network

./asn_capture.py flow -f "ip and net 10.0.0.0/8" -h 10.2.146.10 -p 9000 WARNING: No route found for IPv6 destination :: (no default route?) AS5408 owned by Greek Research and Technology Network S.A: 83.212.7.86 child of 83.212.0.0/16 AS5470 owned by Aristotle University of Thessaloniki: 155.207.113.227 child of 155.207.0.0/16 Traceback (most recent call last): File "./asn_capture.py", line 240, in main() File "./asn_capture.py", line 233, in main callback=parse_packet) File "./asn_capture.py", line 162, in netflow_v5_capture returned = callback(packet) File "./asn_capture.py", line 201, in parse_packet dst_owner = dst.orgname File "/root/.local/lib/python2.7/site-packages/asn_report/lookup.py", line 88, in getattr self.data[attr] = self._lookup(attr) File "/root/.local/lib/python2.7/site-packages/asn_report/lookup.py", line 126, in _lookup return self.getattribute('lookup' + keyword)() File "/root/.local/lib/python2.7/site-packages/asn_report/lookup.py", line 157, in _lookup_orgname return self._maxmind_org_db['AS%d' % self.asnum] TypeError: %d format: a number is required, not str

I was hopping to avoid poking with Maxmind format and perhaps put the org description as a third field on the ip_to_asn.db file or perhaps just use the prefix as ASN description

Edit: I am in the process of converting all this to a Maxmind GeoIPASNum2.csv :)

[acoul]

here is a sample of my custom tab separated ip_to_asn.db 10.0.11.0/24 21338 10.2.10.0/24 3298 10.2.1.0/24 16924 10.2.102.0/24 2822 10.2.107.0/24 20409

and here is the custom of GeoIPASNum2.csv: 170211329,170211582,"AS1 dti" 173521665,173521918,"AS4 aangelis" 169850113,169850366,"AS19 m0rales" 172607745,172607998,"AS21 dti-21" 171041281,171041534,"AS24 dalex"

issuing a ./asn_capture.py flow -f 'net 10.0.0.0/8' -h 10.2.146.10 -p 9000 results in:

WARNING: No route found for IPv6 destination :: (no default route?) AS15169 owned by Google Inc.: 216.58.212.174 child of 216.58.212.0/24 AS15169 owned by Google Inc.: 216.58.211.2 child of 216.58.211.0/24 AS15169 owned by Google Inc.: 74.125.195.189 child of 74.125.195.0/24 AS15169 owned by Google Inc.: 216.58.212.202 child of 216.58.212.0/24 AS6799 owned by Ote SA (Hellenic Telecommunications Organisation): 62.103.68.175 child of 62.103.0.0/16 AS12713 owned by OTE GLOBAL SOLUTIONS S.A.: 62.75.23.217 child of 62.75.23.0/24

am I missing something in PCAP filter syntax ?

Edit: forcing pmacctd/netflow_v5 on sending only 10.0.0.0/8 data results in null/empty reports. any clues ?

[acoul]

a few questions that will welcome some affection:

Why do we need to reinstall every time that we may change/update the two offline databases? How adding them while not committing them to the git tree will make setuptools notices them?

Does commit If non-public IP is encountered, just ignore and continue on has anything to do with my ultimate failures on making this work on a private AS/Prefix network?

so far <<./asn_capture.py pcap -f "ip and net 10.0.0.0/8">> results in absolute silence

while <<./asn_capture.py pcap>> works for a few seconds on ?PUBLIC? networks and then spits out the following:

AS15169 owned by Google Inc.: 74.125.195.132 child of 74.125.195.0/24 AS3329 owned by Vodafone-panafon Hellenic Telecommunications Company SA: 62.38.6.141 child of 62.38.0.0/16 AS29990 owned by AppNexus: 185.33.220.210 child of 185.33.220.0/22 AS29990 owned by AppNexus: 185.33.220.210 child of 185.33.220.0/22 AS3329 owned by Vodafone-panafon Hellenic Telecommunications Company SA: 62.38.6.140 child of 62.38.0.0/16 Traceback (most recent call last): File "./asn_capture.py", line 240, in main() File "./asn_capture.py", line 221, in main sniff(filter=scapy_filter, prn=parse_packet) File "/root/.local/lib/python2.7/site-packages/scapy/sendrecv.py", line 620, in sniff r = prn(p) File "./asn_capture.py", line 201, in parse_packet dst_owner = dst.orgname File "/root/.local/lib/python2.7/site-packages/asn_report/lookup.py", line 88, in getattr self.data[attr] = self._lookup(attr) File "/root/.local/lib/python2.7/site-packages/asn_report/lookup.py", line 126, in _lookup return self.getattribute('lookup' + keyword)() File "/root/.local/lib/python2.7/site-packages/asn_report/lookup.py", line 157, in _lookup_orgname return self._maxmind_org_db['AS%d' % self.asnum] TypeError: %d format: a number is required, not str

Edit: updating to pyasn-git and to latest IPASN data file makes things better but apparently when an AS/Prefix is not found asn_report/lookup.py explodes

Still looking for some help/feedback on the private AS/Prefix quest, maybe ipasn.fake

[coxley]

Putting the code outputs in code-blocks would make this way easier to quickly mind-parse. :)

Yeah I imagine a lot of the code here needs some clean-up, it's been a couple years. I've another project coxley/aspath_graph that lets (or rather requires :P) users to provide custom AS-to-name mapping. Could probably do the same here.

Willing to accept PRs ;)

[acoul]

You mean this along with another pending ticket by me.

Both projects are quite neat and probably unique in the open source land on visualizing what is going on especially on free/open community wireless networks.

[acoul]

problem solved. there were some non-latin characters on the custom GeoIPASNum2.csv. a little print statement revealed the evil that was corrected and since everything works like a charm.

AS22128 owned by SupperQuagga: 10.2.146.10 child of 10.2.146.0/24 AS22128 owned by SupperQuagga: 10.2.146.10 child of 10.2.146.0/24 AS3298 owned by OZOnet: 10.2.19.3 child of 10.2.19.0/24 AS6275 owned by ttel: 10.34.64.47 child of 10.34.64.0/24 AS3298 owned by OZOnet: 10.2.19.3 child of 10.2.19.0/24 AS14073 owned by gpa: 10.3.43.23 child of 10.3.43.0/24 AS3298 owned by OZOnet: 10.2.19.18 child of 10.2.19.0/24 AS3298 owned by OZOnet: 10.2.19.3 child of 10.2.19.0/24 AS3298 owned by OZOnet: 10.2.19.3 child of 10.2.19.0/24

It would be great to have the pcap filter working on netflow collector too.

[acoul]

I had to also change:

-if IP(dst_ip).iptype() is 'PUBLIC': +if IP(dst_ip).iptype() is 'PRIVATE':

now asn_report/lookup.py explodes every time it gets an IP that has no match in the GeoIPASNum2.csv file. If I can only make this a non destructive check.

AS21091 owned by Paralia_Kaloni: 10.90.180.254 child of 10.90.180.0/24 22128 AS22128 owned by SupperQuagga: 10.2.146.10 child of 10.2.146.0/24 AS lookup failed: 192.168.1.69 Traceback (most recent call last): File "./asn_capture.py", line 240, in main() File "./asn_capture.py", line 221, in main sniff(filter=scapy_filter, prn=parse_packet) File "/root/.local/lib/python2.7/site-packages/scapy/sendrecv.py", line 620, in sniff r = prn(p) File "./asn_capture.py", line 201, in parse_packet dst_owner = dst.orgname File "/root/.local/lib/python2.7/site-packages/asn_report/lookup.py", line 88, in getattr self.data[attr] = self._lookup(attr) File "/root/.local/lib/python2.7/site-packages/asn_report/lookup.py", line 126, in _lookup return self.getattribute('lookup' + keyword)() File "/root/.local/lib/python2.7/site-packages/asn_report/lookup.py", line 157, in _lookup_orgname return self._maxmind_org_db['AS%d' % self.asnum] TypeError: %d format: a number is required, not str

Edit: the following looks like is doing the trick:

-return self._maxmind_org_db['AS%d' % self.asnum] +try: +return self._maxmind_org_db['AS%d' % self.asnum] +except Exception: +pass

[acoul]

here is your code live & kicking