sbng / mrt2mmdb

Convert a mrt file into a maxmind database file
1 stars 0 forks source link

Crash - list index out of range #10

Closed johnhtodd closed 3 weeks ago

johnhtodd commented 6 months ago

I get a crash on this version.

I created a new MRT file (ams-20240327.mrt.gz) I downloaded a new CSV of ASN mappings (./data/asn_rir_org_country ) I had previously renamed the Maxmind-supplied ASN mapping file, so I just specified the new name on the command line.

If desired, I can get to you the new MRT file.

Also note: I am using python3 for this (modified first line of make_mmdb.py to read "#!/usr/bin/env python3")

root@dev01:~/mrt/mrt2mmdb/mrt2mmdb# ./make_mmdb.py --mrt data/ams-20240327.mrt.gz --target ASN-Extended.mmdb --lookup_file ./data/asn_rir_org_country --custom_lookup_only --database_type bgp-table --mmdb data/old-GeoLite2-ASN.mmdb
 Making custom ASN table using lookup file : 143752 prefixes [00:00, 456067.93 prefixes/s]
 Loading mrt data into dictionary          : 1133165 prefixes [02:32, 7421.68 prefixes/s]
 Converting mrt into mmda                  : 934067 prefixes [02:06, 7359.58 prefixes/s]
Traceback (most recent call last):
  File "/root/mrt/mrt2mmdb/mrt2mmdb/./make_mmdb.py", line 306, in <module>
    main()
  File "/root/mrt/mrt2mmdb/mrt2mmdb/./make_mmdb.py", line 291, in main
    missing, convert_stats = convert_mrt_mmdb(args.target, prefixes_mrt, asn)
  File "/root/mrt/mrt2mmdb/mrt2mmdb/./make_mmdb.py", line 54, in timeit_wrapper
    result, count = func(*listargs, **kwargs)
  File "/root/mrt/mrt2mmdb/mrt2mmdb/./make_mmdb.py", line 207, in convert_mrt_mmdb
    writer.insert_network(
  File "/usr/local/lib/python3.10/dist-packages/mmdb_writer.py", line 446, in insert_network
    current_node[bits[-1]] = leaf
IndexError: list index out of range
root@dev01:~/mrt/mrt2mmdb/mrt2mmdb#
johnhtodd commented 6 months ago

I neglected to say: I'm running the "differences" branch.

johnhtodd commented 6 months ago

Though I am now trying older versions and I get the same crash on all of them.

sbng commented 6 months ago

@johnhtodd could u kindly email me the mrt file? Thanks. apparently, there are some entries within the mrt files has no aspath. I can skip those entries but I just need to confirm my diagnostic is correct.