Understanding differences between different BGP route dumps

hadiasghari commented 8 years ago

pyasn can take as input BGP route dumps from different sources. Understanding the differences between these sources can help select the default source.

The download script currently defaults to RouteViews v4 archive.
RouteViews also has a v4+v6 archive that might be a better default, given the increased adoption of IPv6. Interestingly enough, these dumps are typically smaller than the v4 only dump, despite having more prefixes.
Some users have advocated using the RIPE RIS archives (http://data.ris.ripe.net/rrc12/) which are also v4+v6.

Understanding the differences between these MRT route dumps would be useful -- in terms of accuracy and completeness. For this purpose:

A detailed-dump/debug functionality could be added to pyasn_util_convert -- to trace why the v4 only file is larger.
Alternatively, a new script can be written to compares two generated IPASNDATs, and list IP ranges that would resolve to different ASNs.

hadiasghari commented 7 years ago

An update: I've added the mrtx.py/method dump_verbose_mrt_file() as a starting point for printing more details from an RIB file.

hadiasghari commented 7 years ago

The discussion below is moved here from #39. It is on determining the correct origin-AS in an MRT/RIB file when multiple AS-paths are present for one prefix (-- this almost always happens, and we currently pick the first). It relates to differences among multiple RIB files (from different vantage points) as there we also have multiple multiple AS-paths.

@Eichhoernchen: "I readded the part of the code that traverses all peer entries of a table entry... some have inconclusive data added to them, some routers seem advertise false prefixes, at least some of them had multiple AS that pointed to the same prefix. Wouldn't it be better to return a set of ASs in these cases?"

@hadiasghari I don't have a clear answer for your question yet. Ideally we wouldn't want to return all the ASes pointing to the same prefix, but the best one that the BGP router matches (if such a thing makes sense). If you use the latest code from master, there is a new '--dump' switch to 'pyasn_util_convert.py' to help with debugging this scenario.

Sample output:

$ pyasn_util_convert.py --dump rib.20170102.1400.bz2 --record-to 5
Dumping MRT/RIB archive to screen:

Record #000001: MrtTD2Record (PEER-INDEX-TABLE, collector 2162111334, 68 peers)

Record #000002: MrtTD2Record (IPV4-UNICAST 0.0.0.0/0, 6 entries)
     Entry 01 BGPAttribute(ORIGIN): 0
              BGPAttribute(AS_PATH): path-sequence[34224, 3257]
              BGPAttribute(NEXT_HOP): 1587346450
              BGPAttribute(MULTI_EXIT_DISC): 0
              BGPAttribute(COMMUNITIES): 2242904397
     Entry 02 BGPAttribute(ORIGIN): 0
              BGPAttribute(AS_PATH): path-sequence[18106]
              BGPAttribute(NEXT_HOP): 3393792045
     Entry 03 BGPAttribute(ORIGIN): 0
              BGPAttribute(AS_PATH): path-sequence[20771, 3356]
              BGPAttribute(NEXT_HOP): 1358016543
     Entry 04 BGPAttribute(ORIGIN): 0
              BGPAttribute(AS_PATH): path-sequence[31019, 39326]
              BGPAttribute(NEXT_HOP): 1541707521
     Entry 05 BGPAttribute(ORIGIN): 0
              BGPAttribute(AS_PATH): path-sequence[58511]
              BGPAttribute(NEXT_HOP): 1744241453
              BGPAttribute(MULTI_EXIT_DISC): 0
     Entry 06 BGPAttribute(ORIGIN): 0
              BGPAttribute(AS_PATH): path-sequence[47872, 3356]
              BGPAttribute(NEXT_HOP): 3106698241
              BGPAttribute(COMMUNITIES): 20 bytes
     => pyasn choice: AS 3257

Record #000003: MrtTD2Record (IPV4-UNICAST 1.0.0.0/24, 1 entries)
     Entry 01 BGPAttribute(ORIGIN): 0
              BGPAttribute(AS_PATH): path-sequence[20771, 47872]
              BGPAttribute(NEXT_HOP): 1358016543
     => pyasn choice: AS 47872

Record #000004: MrtTD2Record (IPV4-UNICAST 1.0.4.0/24, 42 entries)
     Entry 01 BGPAttribute(ORIGIN): 0
              BGPAttribute(AS_PATH): path-sequence[8492, 20764, 3216, 4637, 1221, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 56203]
              BGPAttribute(NEXT_HOP): 1433534681
              BGPAttribute(COMMUNITIES): 40 bytes
Entry 02 BGPAttribute(ORIGIN): 0
              BGPAttribute(AS_PATH): path-sequence[1239, 4637, 4637, 4637, 4637, 1221, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 56203]
              BGPAttribute(NEXT_HOP): 2430923138
              BGPAttribute(MULTI_EXIT_DISC): 225
     Entry 03 BGPAttribute(ORIGIN): 0
              BGPAttribute(AS_PATH): path-sequence[6762, 4637, 1221, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 56203]
              BGPAttribute(NEXT_HOP): 3273054396
              BGPAttribute(MULTI_EXIT_DISC): 100
              BGPAttribute(COMMUNITIES): 443154463
     Entry 04 BGPAttribute(ORIGIN): 0
              BGPAttribute(AS_PATH): path-sequence[53364, 3257, 4637, 1221, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 56203]
              BGPAttribute(NEXT_HOP): 2915908074
              BGPAttribute(COMMUNITIES): 20 bytes
     Entry 05 BGPAttribute(ORIGIN): 0
              BGPAttribute(AS_PATH): path-sequence[34224, 5580, 4637, 1221, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 56203]
              BGPAttribute(NEXT_HOP): 1587346450
              BGPAttribute(MULTI_EXIT_DISC): 0
              BGPAttribute(COMMUNITIES): 12 bytes
     Entry 06 BGPAttribute(ORIGIN): 0
              BGPAttribute(AS_PATH): path-sequence[1668, 4637, 1221, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 56203]
              BGPAttribute(NEXT_HOP): 1119453185
              BGPAttribute(MULTI_EXIT_DISC): 162
     Entry 07 BGPAttribute(ORIGIN): 0
              BGPAttribute(AS_PATH): path-sequence[7018, 209, 4637, 1221, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 38803, 56203]
              BGPAttribute(NEXT_HOP): 201326911
              BGPAttribute(COMMUNITIES): 8 bytes
<CUT>

hadiasghari commented 7 years ago

@Eichhoernchen, do you have example of specific prefixes with problems that we can delve into? Thanks.

janrueth commented 7 years ago

Sure, for example Prefix: 38.67.212.0/22 (at least in data from end of last year and start of this year) It is announced in AS174 and AS394540. The odd thing is that all announcements that lead to AS394540 use BGP prepending, which seems not to really make sense to me given that AS394540 is the destination, if the AS in front of AS394540 were to duplicate its entry that would make sense to me but this way seems rather confusing to me.

hadiasghari / pyasn

Understanding differences between different BGP route dumps #37