search

svaningelgem / location_ipfire_db_reader

MIT License
0 stars 0 forks source link

Just had a fun with you project but looks wrong. #1

Closed MaurUppi closed 8 months ago

MaurUppi commented 8 months ago

ip_location_script.py kicked off and the location.db will be downloaded in the same folder. 


import sys
from pathlib import Path
from location_ipfire_db_reader import LocationDatabase

def main(ip_address):
    location_db = Path('location.db')
    # This call will download the location database into the provided file.
    # It will not re-download it if there are no updates.
    LocationDatabase.download(location_db)

    db = LocationDatabase(location_db)
    country = db.find_country(ip_address)
    print(f'The country for IP {ip_address} is: {country}')

if __name__ == "__main__":
    if len(sys.argv) != 2:
        print("Usage: python script.py <IP_ADDRESS>")
        sys.exit(1)

    ip_address = sys.argv[1]
    main(ip_address)

However, the lookup result is not good. 

PS D:\Github\location_ipfire_db_reader\src\location_ipfire_db_reader> python.exe .\ip_location_script.py 8.8.8.8
The country for IP 8.8.8.8 is:
PS D:\Github\location_ipfire_db_reader\src\location_ipfire_db_reader> python.exe .\ip_location_script.py 1.1.1.1
The country for IP 1.1.1.1 is: AU
PS D:\Github\location_ipfire_db_reader\src\location_ipfire_db_reader> python.exe .\ip_location_script.py 1.32.233.111
The country for IP 1.32.233.111 is: AU
PS D:\Github\location_ipfire_db_reader\src\location_ipfire_db_reader> python.exe .\ip_location_script.py 1.32.252.111
The country for IP 1.32.252.111 is: AU
PS D:\Github\location_ipfire_db_reader\src\location_ipfire_db_reader> python.exe .\ip_location_script.py 113.74.8.78
The country for IP 113.74.8.78 is: AU
PS D:\Github\location_ipfire_db_reader\src\location_ipfire_db_reader> python.exe .\ip_location_script.py 212.107.28.52
The country for IP 212.107.28.52 is: AU
svaningelgem commented 8 months ago

Thanks for the report.

I indeed see that something is going wrong:

Expected :'US'
Actual   :'\x00\x00'
svaningelgem commented 8 months ago

Hi @MaurUppi ,

Thanks for your report. I think I fixed the code now. So once you update the library with:

pip install -U location-ipfire-db-reader

You should be getting 1.0.4, which will have the expected outputs.

I also changed the API, so it's much simpler:

from location_ipfire_db_reader import LocationDatabase

# This call will _download the location database into the provided file.
#   It will not re-_download it if there are no updates.
db = LocationDatabase('location.db')
print(db.find_country('8.8.8.8'))  # US

This way you don't need to add the additional download-step, and you'll stay up-to-date automatically during usage (maximum checks once a day).

If you find anything else, please feel free to re-open and/or raise another issue.

Grtz, Steven

MaurUppi commented 8 months ago

refined ip_location_script.py and it runs as expected now. 

import sys
from location_ipfire_db_reader import LocationDatabase

def main(ip_address):
    db = LocationDatabase('location.db')
    country = db.find_country(ip_address)
    print(f'The country for IP {ip_address} is: {country}')

if __name__ == "__main__":
    if len(sys.argv) != 2:
        print("Usage: python script.py <IP_ADDRESS>")
        sys.exit(1)

    ip_address = sys.argv[1]
    main(ip_address)
PS D:\Github\location_ipfire_db_reader\src\location_ipfire_db_reader> python.exe .\ip_location_script.py 212.107.28.52
The country for IP 212.107.28.52 is: AP
PS D:\Github\location_ipfire_db_reader\src\location_ipfire_db_reader> python.exe .\ip_location_script.py 1.32.252.111
The country for IP 1.32.252.111 is: AP
PS D:\Github\location_ipfire_db_reader\src\location_ipfire_db_reader> python.exe .\ip_location_script.py 113.74.8.78
The country for IP 113.74.8.78 is: CN
PS D:\Github\location_ipfire_db_reader\src\location_ipfire_db_reader> python.exe .\ip_location_script.py 1.1.1.1
The country for IP 1.1.1.1 is: AU
PS D:\Github\location_ipfire_db_reader\src\location_ipfire_db_reader> python.exe .\ip_location_script.py 1.32.233.111
The country for IP 1.32.233.111 is: US

BTW, given the location.db also included AS info, I am just curious to ask if consider including a feature to output such info?

aut-num:                 AS198871
name:                    Diputacion Provincial de Castellon

net:                     5.39.208.0/22
country:                 ES
aut-num:                 198871

net:                     5.39.208.0/24
country:                 ES
aut-num:                 198871

net:                     5.39.209.0/24
country:                 ES
aut-num:                 198871

net:                     5.39.210.0/24
country:                 ES
aut-num:                 198871
svaningelgem commented 8 months ago

Yes, there is more information available in there. What would your proposal be to output?

I was thinking something like:

db = LocationDatabase('location.db')
db["1.2.3.4"] # would return a class with all available information for this like country, asn, names, ... ?

What do you think?

MaurUppi commented 8 months ago

Yes, there is more information available in there. What would your proposal be to output?

I was thinking something like:

db = LocationDatabase('location.db')
db["1.2.3.4"] # would return a class with all available information for this like country, asn, names, ... ?

What do you think?

yeah, I think outputting what network fits is totally ok. 

  "aut-num":  AS#
  "name":  ASorg
  "country": US/VN/CA etc. 
  "is-anycast": YES/NO
  "is-satellite-provider": YES/NO
  "is-anonymous-proxy": YES/NO
svaningelgem commented 8 months ago

I added a bit more this morning: README.md

wdyt?

MaurUppi commented 8 months ago

I added a bit more this morning: README.md

wdyt?

It Looks like pip has not yet upstreamed. Am I wrong?

PS D:\Github\location_ipfire_db_reader> pip install -U location-ipfire-db-reader
Requirement already satisfied: location-ipfire-db-reader in c:\users\nucrpl\appdata\local\programs\python\python312\lib\site-packages (1.0.4)
Requirement already satisfied: requests>=2 in c:\users\nucrpl\appdata\local\programs\python\python312\lib\site-packages (from location-ipfire-db-reader) (2.31.0)
Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\nucrpl\appdata\local\programs\python\python312\lib\site-packages (from requests>=2->location-ipfire-db-reader) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in c:\users\nucrpl\appdata\local\programs\python\python312\lib\site-packages (from requests>=2->location-ipfire-db-reader) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in c:\users\nucrpl\appdata\local\programs\python\python312\lib\site-packages (from requests>=2->location-ipfire-db-reader) (2.1.0)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\nucrpl\appdata\local\programs\python\python312\lib\site-packages (from requests>=2->location-ipfire-db-reader) (2023.11.17)

RECORD

location_ipfire_db_reader-1.0.4.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4
location_ipfire_db_reader-1.0.4.dist-info/METADATA,sha256=r78cMXP2B6hTGa1AUA59GLVSaqfU-cZhS_nYAP5BcjU,2009
location_ipfire_db_reader-1.0.4.dist-info/RECORD,,
location_ipfire_db_reader-1.0.4.dist-info/REQUESTED,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
location_ipfire_db_reader-1.0.4.dist-info/WHEEL,sha256=FMvqSimYX_P7y0a7UY-_Mc83r5zkBZsCYPm7Lr0Bsq4,88
location_ipfire_db_reader/__init__.py,sha256=dCAUmK-G10lfm1DInkSvUBUKVSDWl1PqNsEKE8fvk5k,74
location_ipfire_db_reader/__pycache__/__init__.cpython-312.pyc,,
location_ipfire_db_reader/__pycache__/database.cpython-312.pyc,,
location_ipfire_db_reader/__pycache__/decompress_db.cpython-312.pyc,,
location_ipfire_db_reader/__pycache__/download_db.cpython-312.pyc,,
location_ipfire_db_reader/__pycache__/interpret_location_db.cpython-312.pyc,,
location_ipfire_db_reader/database.py,sha256=0l1qJGvLaG3qSWXfkUkEqvVHCG4u6ggV6BDXRGm2tU8,4211
location_ipfire_db_reader/decompress_db.py,sha256=cVngny-os3FHjTf9x6BYkmW7uzDlpHSsWBYP_lM3eqE,547
location_ipfire_db_reader/download_db.py,sha256=NBNsDUGUC6oNOZ9MSCK2o4aPvb76BuLVRlR0vy7MZU4,1628
location_ipfire_db_reader/interpret_location_db.py,sha256=3A1qZAdUljh8ykw-tBhFj4LZ9_KekWl6E5zR2ZY4EzE,10114

ip_information.py is NOT there.

PS D:\Github\location_ipfire_db_reader> git pull
remote: Enumerating objects: 44, done.
remote: Counting objects: 100% (44/44), done.
remote: Compressing objects: 100% (28/28), done.
remote: Total 39 (delta 15), reused 32 (delta 10), pack-reused 0
Unpacking objects: 100% (39/39), 20.64 KiB | 681.00 KiB/s, done.
From https://github.com/svaningelgem/location_ipfire_db_reader
   76208a5..2a87506  master     -> origin/master
 * [new branch]      output-everything-about-an-ip -> origin/output-everything-about-an-ip
Updating 76208a5..2a87506
Fast-forward
 .gitignore                                         |   1 +
 README.md                                          |  36 +-
 .../__init__.py                                    |   0
 location_ipfire_db_reader/database.py              | 127 ++++++
 .../decompress_db.py                               |   8 +-
 location_ipfire_db_reader/download_db.py           |  49 +++
 .../interpret_location_db.py                       |  15 +-
 poetry.lock                                        | 459 ++++++++-------------
 pyproject.toml                                     |  34 +-
 src/location_ipfire_db_reader/database.py          | 112 -----
 src/location_ipfire_db_reader/download_db.py       |  45 --
 tests/conftest.py                                  |  13 +
 tests/test_issue_1.py                              |  24 ++
 13 files changed, 456 insertions(+), 467 deletions(-)
 rename {src/location_ipfire_db_reader => location_ipfire_db_reader}/__init__.py (100%)
 create mode 100644 location_ipfire_db_reader/database.py
 rename {src/location_ipfire_db_reader => location_ipfire_db_reader}/decompress_db.py (79%)
 create mode 100644 location_ipfire_db_reader/download_db.py
 rename {src/location_ipfire_db_reader => location_ipfire_db_reader}/interpret_location_db.py (96%)
 delete mode 100644 src/location_ipfire_db_reader/database.py
 delete mode 100644 src/location_ipfire_db_reader/download_db.py
 create mode 100644 tests/conftest.py
 create mode 100644 tests/test_issue_1.py
PS D:\Github\location_ipfire_db_reader\location_ipfire_db_reader> dir

    Directory: D:\Github\location_ipfire_db_reader\location_ipfire_db_reader

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----         1/21/2024   9:30 AM           4211 database.py
-a----         1/21/2024   9:30 AM            547 decompress_db.py
-a----         1/21/2024   9:30 AM           1628 download_db.py
-a----         1/21/2024   9:30 AM          10114 interpret_location_db.py
-a----         1/21/2024   9:23 AM            882 ip_location_script.py
-a----         1/17/2024  11:13 PM       60159962 location.db
-a----         1/21/2024   9:30 AM             74 __init__.py

Did you push ip_information.py from local to GitHub?

ip_location_script.py

import sys
from location_ipfire_db_reader import LocationDatabase

def main(ip_address):
    db = LocationDatabase('location.db')
    ip_info = db[ip_address]
    print(f"""
    IP: {ip_info.ip}
    Subnet Mask: {ip_info.subnet_mask}
    Network Address: {ip_info.network_address}
    IP with CIDR: {ip_info.ip_with_cidr}

    ASN: {ip_info.asn}
    ASN Name: {ip_info.asn_name}
    Country Code: {ip_info.country_code}
    Country Name: {ip_info.country_name}
    Country Continent: {ip_info.country_continent}

    Is Anonymous Proxy: {ip_info.is_anonymous_proxy}
    Is Satellite Provider: {ip_info.is_satellite_provider}
    Is Anycast: {ip_info.is_anycast}
    Is Drop: {ip_info.is_drop}
    """)

if __name__ == "__main__":
    if len(sys.argv) != 2:
        print("Usage: python script.py <IP_ADDRESS>")
        sys.exit(1)

    ip_address = sys.argv[1]
    main(ip_address)
PS D:\Github\location_ipfire_db_reader\location_ipfire_db_reader> python.exe .\ip_location_script.py 1.32.252.111
Traceback (most recent call last):
  File "D:\Github\location_ipfire_db_reader\location_ipfire_db_reader\ip_location_script.py", line 31, in <module>
    main(ip_address)
  File "D:\Github\location_ipfire_db_reader\location_ipfire_db_reader\ip_location_script.py", line 6, in main
    ip_info = db[ip_address]
              ~~^^^^^^^^^^^^
TypeError: 'LocationDatabase' object is not subscriptable
svaningelgem commented 8 months ago

Yes, I didn't push it yet as I was waiting for your reply to see if you still consider something missing and/or would have a better syntax?

If you think it's fine, I'll upload the new version.

FYI, the branch it's located in is output-everything-about-an-ip.

svaningelgem commented 8 months ago

edit: I uploaded the new version (1.0.5). So you should be able to get it directly from PyPi now.

MaurUppi commented 8 months ago

edit: I uploaded the new version (1.0.5). So you should be able to get it directly from PyPi now.

it works as expected with some IPs!!!

PS D:\Github\location_ipfire_db_reader\location_ipfire_db_reader> python.exe .\ip_location_script.py 1.32.251.111

    IP: 1.32.251.111
    Subnet Mask: 24
    Network Address: 1.32.251.0
    IP with CIDR: 1.32.251.0/24

    ASN: 64050
    ASN Name: BGP Consultancy Pte Ltd
    Country Code: AP
    Country Name: Asia/Pacific
    Country Continent: AS

    Is Anonymous Proxy: False
    Is Satellite Provider: False
    Is Anycast: False
    Is Drop: False

PS D:\Github\location_ipfire_db_reader\location_ipfire_db_reader> python.exe .\ip_location_script.py 113.74.8.78

    IP: 113.74.8.78
    Subnet Mask: 11
    Network Address: 113.64.0.0
    IP with CIDR: 113.64.0.0/11

    ASN: 4134
    ASN Name: Chinanet Backbone
    Country Code: CN
    Country Name: China
    Country Continent: AS

    Is Anonymous Proxy: False
    Is Satellite Provider: False
    Is Anycast: False
    Is Drop: False

PS D:\Github\location_ipfire_db_reader\location_ipfire_db_reader> python.exe .\ip_location_script.py 212.107.28.52

    IP: 212.107.28.52
    Subnet Mask: 24
    Network Address: 212.107.28.0
    IP with CIDR: 212.107.28.0/24

    ASN: 41378
    ASN Name: Kirino LLC
    Country Code: AP
    Country Name: Asia/Pacific
    Country Continent: AS

    Is Anonymous Proxy: False
    Is Satellite Provider: False
    Is Anycast: False
    Is Drop: False

But I'm not sure what happened with 201.148.95.249

http://ip-api.com/csv/201.148.95.249

success,Mexico,MX,CMX,Mexico City,Mexico City,01010,19.3531,-99.2091,America/Mexico_City,"Operbes, S.A. de C.V.","Operbes, S.A. de C.V","AS18734 Operbes, S.A. de C.V.",201.148.95.249
PS D:\Github\location_ipfire_db_reader\location_ipfire_db_reader> python.exe .\ip_location_script.py 201.148.95.249
Traceback (most recent call last):
  File "D:\Github\location_ipfire_db_reader\location_ipfire_db_reader\ip_location_script.py", line 31, in <module>
    main(ip_address)
  File "D:\Github\location_ipfire_db_reader\location_ipfire_db_reader\ip_location_script.py", line 14, in main
    ASN Name: {ip_info.asn_name}
               ^^^^^^^^^^^^^^^^
  File "C:\Users\nucrpl\AppData\Local\Programs\Python\Python312\Lib\functools.py", line 995, in __get__
    val = self.func(instance)
          ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\nucrpl\AppData\Local\Programs\Python\Python312\Lib\site-packages\location_ipfire_db_reader\ip_information.py", line 95, in asn_name
    obj = self._find_object(
          ^^^^^^^^^^^^^^^^^^
  File "C:\Users\nucrpl\AppData\Local\Programs\Python\Python312\Lib\site-packages\location_ipfire_db_reader\ip_information.py", line 84, in _find_object
    raise ValueError(f"Can't find asn object with id {self.asn}!")
ValueError: Can't find asn object with id 18734!
svaningelgem commented 8 months ago

Hi @MaurUppi , I splitted off this latest one into another ticket so it's more visible.