Fixes usage of maxmind geopip db

[Te-k]

It is no longer available without registration : https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-using-geolite2-databases/

The best way is likely to rely on geoipupdate and check the path https://github.com/maxmind/geoipupdate/releases

[Te-k]

Best solution would be :

• Ask to install geoipupdate and configure it https://github.com/maxmind/geoipupdate/releases (configuration from https://www.maxmind.com/en/accounts/161197/license-key/GeoIP.conf )
• Check if geoipupdate exist
• Check when the files where downloaded last, if not recent run it
• Then use the files stored in /var/lib/GeoIP

[joaociocca]

is this why config -u breaks on Downloading MaxMind GeoIP Database?

harpoon config -u
Updating all plugins data:
Updating plugin asn
Downloading CAIDA AS Classification (https://www.caida.org/data/as-classification/)
-/root/.config/harpoon/2015_caida.csv
Updating plugin ip
Downloading MaxMind GeoIP Database
Traceback (most recent call last):
  File "/usr/lib/python3.8/urllib/request.py", line 1319, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.8/http/client.py", line 1230, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1276, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1225, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1004, in _send_output
    self.send(msg)
  File "/usr/lib/python3.8/http/client.py", line 944, in send
    self.connect()
  File "/usr/lib/python3.8/http/client.py", line 915, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.8/socket.py", line 787, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/usr/lib/python3.8/socket.py", line 918, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/harpoon", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/harpoon/main.py", line 61, in main
    plugins[args.command].run(config, args, plugins)
  File "/usr/local/lib/python3.8/dist-packages/harpoon/commands/config.py", line 68, in run
    plugins[p].update()
  File "/usr/local/lib/python3.8/dist-packages/harpoon/commands/ip.py", line 89, in update
    file_name, headers = urllib.request.urlretrieve('http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz')
  File "/usr/lib/python3.8/urllib/request.py", line 247, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1348, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.8/urllib/request.py", line 1322, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>

[Te-k]

Yes, I have to fix that soon

[Te-k]

Fixed, can you try with version 0.1.4 @joaociocca ?

[joaociocca]

config -u done! thank you, @Te-k

[joaociocca]

Could this other error be related?

kali@kali:~/harpoon$ sudo harpoon ip info <ip>
Traceback (most recent call last):
  File "/usr/local/bin/harpoon", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/harpoon/main.py", line 61, in main
    plugins[args.command].run(config, args, plugins)
  File "/usr/local/lib/python3.8/dist-packages/harpoon/commands/ip.py", line 171, in run
    ipinfo = self.ipinfo(ip)
  File "/usr/local/lib/python3.8/dist-packages/harpoon/commands/ip.py", line 132, in ipinfo
    citydb = geoip2.database.Reader(self.geocity)
  File "/usr/lib/python3/dist-packages/geoip2/database.py", line 86, in __init__
    self._db_reader = maxminddb.open_database(fileish, mode)
  File "/usr/lib/python3/dist-packages/maxminddb/__init__.py", line 39, in open_database
    return maxminddb.extension.Reader(database)
FileNotFoundError: No such file or directory: '/usr/share/GeoIP/GeoLite2-City.mmdb'

[Te-k]

Yes, because of changes in MaxMind for geolocation, you now need to install geoipupdate, confiture it correctly and run it to have the last updated databases, and harpoon is using that files (make sure you to have GeoLite2-Country GeoLite2-City GeoLite2-ASN as EditionIDs).

[Te-k]

Is it solved for you ? Let me know if the doc is not clear, I will update that

[joaociocca]

sorry for the delay! it's fixed over here, thanks!

[Te-k]

Cool !