Requests are not processed properly when cloning and deploying the server

[danrvc]

Hi there,

I cloned the repo and deployed the webapp in multiple environments and it's stuck processing the request. Using the library without the webapp works fine though.

[elceef]

My magic crystal ball is discharged at the moment. Can't help you unless you provide some specifics.

[danrvc]

There's no specifics it sends the response with the remaining domain and it's stuck at the same number of remaining domains.

[danrvc]

@elceef You can just try to clone the repo and run webapp.py (in python 3.8/+) in see for yourself.

[elceef]

$ python3 --version
Python 3.8.10
$ ./webapp.py 
 * Serving Flask app "webapp" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:8000/ (Press CTRL+C to quit)

[danrvc]

@elceef Yeah that's what I've done. Running on Python 3.8 in Ubuntu 20.04. It's stuck on the same amount of remaining domains. I've tried letting it run for 2 hours and it doesn't resolve.

[elceef]

Fetch the latest code from Github and run the web app with DEBUG environment variable:

$ DEBUG=1 ./webapp.py

Any errors reported?

[danrvc]

@elceef Alright, I know what's the problem. The requirements.txt file is missing some dependencies. It should be:

GeoIP>=1.3.2
geoip2>=4.0.0
dnspython>=1.16.0
ssdeep>=3.1
ppdeep>=20200505
py-tlsh>=4.5.0
whois>=0.8
tld>=0.9.1
idna>=2.8
selenium>=4.8.3
Pillow>=9.5.0

And there are some libraries that need to be installed: sudo apt-get install libgeoip-dev libffi-dev libfuzzy-dev libfuzzy2

Then everything's working properly.

Maybe worth to update the README.md and add a requirements/dependencies section.

[elceef]

The webapp.py script requires DNSPython and either GeoIP or geoip2. I think you were missing one of the latter.

[elceef]

I've just updated the script to not enforce certain features if depending modules are not present.