sbng / mrt2mmdb

Convert a mrt file into a maxmind database file
1 stars 0 forks source link

New feature to check the difference between Maxmind mmdb and mrt generated mmdb #8

Open sbng opened 3 months ago

sbng commented 3 months ago
$ ./difference.py -h                                                                               [!differences]
usage: difference.py [-h] [--compare_asn] [--mmdb] [--lookup_file] [--log_level] [--compare_routing [...]] [--quiet]

options:
  -h, --help            show this help message and exit
  --compare_asn         Compare the ASN in a mmdb file vs tsv,csv file [filename1] [filename2]
  --mmdb                Filename of Maxmind mmdb file for prefixes lookup and return description/ASN
  --lookup_file         Filename of csv,tsv file for custom ASN lookup
  --log_level           logging level [CRITICAL|WARNING|INFO|DEBUG](default: WARNING)
  --compare_routing [ ...]
                        Compare routing differenceis between [filename1] [filename2] (both files are in mmdb format)
  --quiet               Turn off verbose (default:verbose)
sbng commented 3 months ago

if used in conjunction with --compare_asn and --mmdb, --lookup_file, it'll show the difference between the custom lookup file AS description vs the Maxmind mmdb.

sbng commented 3 months ago
$ python difference.py --compare_asn --mmdb data/GeoLite2-ASN.mmdb --lookup data/asn_rir_org.tsv --quiet > /tmp/out.txt
$ head /tmp/output.txt                                                                             [!differences]
{
 "root['1']": {
  "new_value": "LEVEL 3 PARENT LLC",
  "old_value": "LVLT-1"
 },
 "root['2']": {
  "new_value": "UNIV OF DELAWARE",
  "old_value": "UDEL-DCN"
 },
sbng commented 3 months ago
$ python difference.py --compare_routing data/out_mrtparse.mmdb data/GeoLite2-ASN.mmdb --quiet > /tmp/out.txt
$ head /tmp/out.txt                                                                                
{
 "root['1.6.219.0/24']": {
  "new_value": "137130",
  "old_value": "9583"
 },
 "root['2.20.32.0/22']": {
  "new_value": "20940",
  "old_value": "16625"
 },
sbng commented 3 months ago

redo the output to provide only summary of diff (add, remove or update). For detail out comparison in json format, please use --print_changes. --quiet switch can be used to suppress output in order to allow the json to be pipe to jq.

sbng commented 3 months ago
$ ./difference.py -h                                                                               [!differences]
usage: difference.py [-h] [--compare_asn] [--mmdb] [--lookup_file] [--log_level] [--compare_routing [...]] [--print_changes] [--quiet]

options:
  -h, --help            show this help message and exit
  --compare_asn         Compare the ASN in a mmdb file vs tsv,csv file [filename1] [filename2]
  --mmdb                Filename of Maxmind mmdb file for prefixes lookup and return description/ASN
  --lookup_file         Filename of csv,tsv file for custom ASN lookup
  --log_level           logging level [CRITICAL|WARNING|INFO|DEBUG](default: WARNING)
  --compare_routing [ ...]
                        Compare routing differenceis between [filename1] [filename2] (both files are in mmdb format)
  --print_changes       Print changes when comparing [filename1] [filename2]
  --quiet               Turn off verbose (default:verbose)