Updated Neighbors VS Rules Stats considering ASes in ASRelDB but not IRR

[SichangHe]

-1 for neighbor if no data in AS Relationship DB. -1 for import/export if no data in IRR.

Polars output.

```elixir shape: (95_911, 4) ┌─────────┬──────────┬────────┬────────┐ │ aut_num ┆ neighbor ┆ import ┆ export │ │ --- ┆ --- ┆ --- ┆ --- │ │ u64 ┆ i32 ┆ i32 ┆ i32 │ ╞═════════╪══════════╪════════╪════════╡ │ 37591 ┆ 2 ┆ -1 ┆ -1 │ │ 204753 ┆ 1 ┆ 3 ┆ 3 │ │ 9689 ┆ 2 ┆ 1 ┆ 1 │ │ 62685 ┆ 4 ┆ -1 ┆ -1 │ │ … ┆ … ┆ … ┆ … │ │ 262378 ┆ 7 ┆ 1 ┆ 1 │ │ 139330 ┆ 1 ┆ 0 ┆ 0 │ │ 265651 ┆ 1 ┆ -1 ┆ -1 │ │ 17077 ┆ 10 ┆ 8 ┆ 8 │ └─────────┴──────────┴────────┴────────┘ shape: (9, 5) ┌────────────┬───────────────┬────────────┬───────────┬───────────┐ │ describe ┆ aut_num ┆ neighbor ┆ import ┆ export │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ f64 ┆ f64 ┆ f64 ┆ f64 │ ╞════════════╪═══════════════╪════════════╪═══════════╪═══════════╡ │ count ┆ 95911.0 ┆ 95911.0 ┆ 95911.0 ┆ 95911.0 │ │ null_count ┆ 0.0 ┆ 0.0 ┆ 0.0 ┆ 0.0 │ │ mean ┆ 125671.693403 ┆ 10.15602 ┆ 4.122249 ┆ 4.008946 │ │ std ┆ 112437.127515 ┆ 102.243572 ┆ 52.110393 ┆ 48.272178 │ │ min ┆ 1.0 ┆ -1.0 ┆ -1.0 ┆ -1.0 │ │ 25% ┆ 34821.0 ┆ 1.0 ┆ 0.0 ┆ 0.0 │ │ 50% ┆ 62122.0 ┆ 1.0 ┆ 1.0 ┆ 1.0 │ │ 75% ┆ 205279.5 ┆ 3.0 ┆ 2.0 ┆ 2.0 │ │ max ┆ 6.131644e6 ┆ 9628.0 ┆ 5724.0 ┆ 5344.0 │ └────────────┴───────────────┴────────────┴───────────┴───────────┘ ```

as_neighbors_vs_rules.csv

[SichangHe]

Code used.

```python import pandas as pd import matplotlib.pyplot as plt df = pd.read_csv("as_neighbors_vs_rules.csv") plt.figure(figsize=(15, 15)) plt.scatter(df['neighbor'], df['import'], color='blue', alpha=0.3, s=2, label='Number of Import') plt.scatter(df['neighbor'], df['export'], color='red', alpha=0.3, s=2, label='Number of Export') plt.xlabel('Neighbor') plt.ylabel('Counts') plt.title('Neighbor Counts vs Import/Export Counts for Autonomous Systems') plt.legend() plt.grid(True) plt.show() ```

[SichangHe]

16960 ASes appear in ASRelDB but not IRR. 20468 ASes appear in IRR but not ASRelDB. Total: 95911 ASes.

IPython.

```python In [14]: df[(df['import'] == -1) | (df['export'] == -1)] Out[14]: aut_num neighbor import export 0 37591 2 -1 -1 3 62685 4 -1 -1 6 399566 1 -1 -1 16 11471 1 -1 -1 19 14444 2 -1 -1 ... ... ... ... ... 95883 31957 2 -1 -1 95884 52860 1 -1 -1 95885 264164 2 -1 -1 95895 393635 2 -1 -1 95909 265651 1 -1 -1 [16960 rows x 4 columns] In [15]: df[(df['neighbor'] == -1)] Out[15]: aut_num neighbor import export 11 55704 -1 0 0 33 202322 -1 3 3 44 199337 -1 2 2 45 44502 -1 1 1 48 24788 -1 0 0 ... ... ... ... ... 95896 142031 -1 0 0 95899 17902 -1 3 3 95900 28529 -1 0 0 95901 208690 -1 0 0 95905 211088 -1 9 9 [20468 rows x 4 columns] ```

[cunha]

16960 ASes appear in ASRelDB but not IRR.

Wild. Do you mean ~17K ASes have no import/export rules, or that there are no aut-num objects at all?

[SichangHe]

Wild. Do you mean ~17K ASes have no import/export rules, or that there are no aut-num objects at all?

No aut-num in IRR databases we use.

[SichangHe]

whois some of the AS not in IRR.

```sh $ whois -h whois.afrinic.net AS37591 % This is the AfriNIC Whois server. % The AFRINIC whois database is subject to the following terms of Use. See https://afrinic.net/whois/terms % Note: this output has been filtered. % To receive output for a database update, use the "-B" flag. %ERROR:101: no entries found % % No entries found in source AFRINIC. $ whois -h whois.arin.net AS62685 ASNumber: 62685 ASName: ORIONVM ASHandle: AS62685 RegDate: 2013-08-07 Updated: 2013-08-07 Ref: https://rdap.arin.net/registry/autnum/62685 OrgName: OrionVM Inc OrgId: ORION-34 Address: 320 Judah Street Address: Suite 5 City: San Francisco StateProv: CA PostalCode: 94122 Country: US RegDate: 2013-01-22 Updated: 2020-05-26 Comment: Please send spam and abuse reports to abuse@orionvm.com. Comment: Please include detailed information and times in UTC. Ref: https://rdap.arin.net/registry/entity/ORION-34 OrgTechHandle: NOC12797-ARIN OrgTechName: NOC OrgTechPhone: +1-844-674-6686 OrgTechEmail: noc@orionvm.com OrgTechRef: https://rdap.arin.net/registry/entity/NOC12797-ARIN OrgAbuseHandle: NOC12797-ARIN OrgAbuseName: NOC OrgAbusePhone: +1-844-674-6686 OrgAbuseEmail: noc@orionvm.com OrgAbuseRef: https://rdap.arin.net/registry/entity/NOC12797-ARIN OrgNOCHandle: NOC12797-ARIN OrgNOCName: NOC OrgNOCPhone: +1-844-674-6686 OrgNOCEmail: noc@orionvm.com OrgNOCRef: https://rdap.arin.net/registry/entity/NOC12797-ARIN $ whois -h whois.arin.net AS31957 ASNumber: 31957 ASName: AGILE ASHandle: AS31957 RegDate: 2015-07-10 Updated: 2015-07-10 Ref: https://rdap.arin.net/registry/autnum/31957 OrgName: Agile Networks OrgId: ANBL-2 Address: 213 Market Ave N Suite 310 City: Canton StateProv: OH PostalCode: 44702 Country: US RegDate: 2012-10-03 Updated: 2023-08-11 Comment: http://www.agilenetworks.com Ref: https://rdap.arin.net/registry/entity/ANBL-2 OrgTechHandle: SPILL46-ARIN OrgTechName: Spillman, Phillip OrgTechPhone: +1-419-631-8495 OrgTechEmail: phil.spillman@agilenetworks.com OrgTechRef: https://rdap.arin.net/registry/entity/SPILL46-ARIN OrgTechHandle: ROBER1880-ARIN OrgTechName: Robertson, Zane OrgTechPhone: +1-330-806-6246 OrgTechEmail: zane.robertson@agilenetworks.com OrgTechRef: https://rdap.arin.net/registry/entity/ROBER1880-ARIN OrgTechHandle: KREIG3-ARIN OrgTechName: Kreigh, Russ OrgTechPhone: +1-260-446-4094 OrgTechEmail: russ.kreigh@agilenetworks.com OrgTechRef: https://rdap.arin.net/registry/entity/KREIG3-ARIN OrgAbuseHandle: SPILL46-ARIN OrgAbuseName: Spillman, Phillip OrgAbusePhone: +1-419-631-8495 OrgAbuseEmail: phil.spillman@agilenetworks.com OrgAbuseRef: https://rdap.arin.net/registry/entity/SPILL46-ARIN OrgRoutingHandle: SPILL46-ARIN OrgRoutingName: Spillman, Phillip OrgRoutingPhone: +1-419-631-8495 OrgRoutingEmail: phil.spillman@agilenetworks.com OrgRoutingRef: https://rdap.arin.net/registry/entity/SPILL46-ARIN OrgDNSHandle: SPILL46-ARIN OrgDNSName: Spillman, Phillip OrgDNSPhone: +1-419-631-8495 OrgDNSEmail: phil.spillman@agilenetworks.com OrgDNSRef: https://rdap.arin.net/registry/entity/SPILL46-ARIN OrgNOCHandle: SPILL46-ARIN OrgNOCName: Spillman, Phillip OrgNOCPhone: +1-419-631-8495 OrgNOCEmail: phil.spillman@agilenetworks.com OrgNOCRef: https://rdap.arin.net/registry/entity/SPILL46-ARIN OrgTechHandle: HOTTL1-ARIN OrgTechName: Hottle, Jesse OrgTechPhone: +1-888-494-0549 OrgTechEmail: jesse.hottle@agilenetworks.com OrgTechRef: https://rdap.arin.net/registry/entity/HOTTL1-ARIN ```

[SichangHe]

I realized a potential problem, @cunha. We are using the AS Relationship DB from 2013, which is probably outdated.

Maybe we should update to one of CAIDA's other DBs such as as-relationships?

[SichangHe]

No longer latest stats after fixing default.

[SichangHe]

Covered by #95.

[cunha]

I realized a potential problem, @cunha. We are using the AS Relationship DB from 2013, which is probably outdated.

Maybe we should update to one of CAIDA's other DBs such as as-relationships?

We should definitely be using a newer database. Something to match the IRR datasets we're using. Did you do this at one point?

[SichangHe]

No, we are not using databases from 2013. It is just under that folder. It has been undated regularly since and we have been using the July 2023 one.

[cunha]

Sounds good. Just checking.