search

hadiasghari / pyasn

Python IP address to Autonomous System Number lookup module. (Supports fast local lookups, and historical lookups using archived BGP dumps.)
Other
292 stars 72 forks source link

AssertionError (sub_type1 == MrtRecord.T1_AFI_IPv4) when converting RIB #42

Closed johanmazel closed 7 years ago

johanmazel commented 7 years ago

I am trying to convert some RIB data from WIDE: http://archive.routeviews.org/route-views.wide/bgpdata/.

I am getting this assertion error : Traceback (most recent call last): File "/home/jmazel/.local/bin/pyasn_util_convert.py", line 53, in dat = mrtx.parse_mrt_file(f, print_progress=print_progress) File "/home/jmazel/.local/lib/python2.7/site-packages/pyasn/mrtx.py", line 64, in parse_mrt_file mrt = MrtRecord.next_dump_table_record(mrt_file) File "/home/jmazel/.local/lib/python2.7/site-packages/pyasn/mrtx.py", line 217, in next_dump_table_record mrt.table = MrtTableDump1(buf, mrt.sub_type) File "/home/jmazel/.local/lib/python2.7/site-packages/pyasn/mrtx.py", line 260, in init assert sub_type1 == MrtRecord.T1_AFI_IPv4
AssertionError

Files that generate this error are: rib.20040701.0000.bz2 rib.20050101.0132.bz2 rib.20050701.0022.bz2 rib.20060101.0118.bz2 rib.20060701.0051.bz2 rib.20070101.0035.bz2 rib.20070701.0145.bz2 rib.20080101.0106.bz2 rib.20080701.0019.bz2 rib.20090101.0119.bz2 rib.20090701.0141.bz2 rib.20100101.0050.bz2 rib.20100701.0056.bz2

I did not find official md5 hashes of the files so I cannot check if I have the correct file. I however downloaded all of them a second time and I got the same error.

This error may be similar to https://github.com/hadiasghari/pyasn/issues/39 although the failing assert is not the same.

hadiasghari commented 7 years ago

@johanmazel , I checked. This is a separate from the other issue. Could you clarify two things so I can continue the investigation?

  1. Did you run the conversion on all files of this archive (route-views.wide) and got error only on these files? Or did you randomly select these files, and get error on all of them?
  2. How does route-views.wide differ from the normal route-views archive?

Thanks.

johanmazel commented 7 years ago
  1. I actually I run the script on a subset of this archive (route-views.wide): all years on January 1st and July 1st. I can provide the complete file list if you want.
  2. If I am correct, the collection point is located in the WIDE network: http://www.wide.ad.jp/.
hadiasghari commented 7 years ago

@johanmazel , this was related to the RIB files using TableDumpV1 + IPv6, which was originally not implemented. I have fixed this, and pushed it to master. Please have a look, and if there is a problem, reopen the issue. Cheers.

johanmazel commented 7 years ago

I can confirm that it is working now.