Python 3.6 or greater is required.
.. image:: https://img.shields.io/pypi/v/detectem.svg :target: https://pypi.python.org/pypi/detectem
.. image:: https://img.shields.io/travis/alertot/detectem.svg :target: https://travis-ci.org/alertot/detectem
detectem is a specialized software detector. Let's see it in action.
.. code-block:: bash
$ det http://domain.tld [{'name': 'phusion-passenger', 'version': '4.0.10'}, {'name': 'apache-mod_bwlimited', 'version': '1.4'}, {'name': 'apache-mod_fcgid', 'version': '2.3.9'}, {'name': 'jquery', 'version': '1.11.3'}, {'name': 'crayon-syntax-highlighter', 'version': '2.7.2_beta'}]
Using a serie of indicators, it's able to detect software running on a site and extract accurately its version information. It uses Splash API to render the website and start the detection routine. It does full analysis on requests, responses and even on the DOM!
There are two important articles to read:
Reasons to create detectem <http://www.spect.cl/blog/2016/11/challenges-in-web-software-detection/>
_Introduction to detectem <http://www.spect.cl/blog/2016/11/introducing-detectem/>
_Install Docker_ and add your user to the docker group, then you avoid to use sudo.
Pull the image::
$ docker pull scrapinghub/splash
Create a virtual environment with Python >= 3.6 .
Install detectem::
$ pip install detectem
Run it against some URL::
$ det http://domain.tld
Let's see it in action.
.. code-block:: bash
$ docker-compose run --rm detectem http://domain.tld [{'name': 'phusion-passenger', 'version': '4.0.10'}, {'name': 'apache-mod_bwlimited', 'version': '1.4'}, {'name': 'apache-mod_fcgid', 'version': '2.3.9'}, {'name': 'jquery', 'version': '1.11.3'}, {'name': 'crayon-syntax-highlighter', 'version': '2.7.2_beta'}]
But first that all we must do:
Install the last Docker CE Stable version
_.
Add your user to the docker group and logout::
$ sudo usermod -aG docker $USER
Make sure you have logout to apply changes, then log in again.
Install Docker Compose
_
Download to your workspace the docker-compose building files.
Dockerfile-alternate
docker-compose.yml
Build the required docker images for detectem at the same directory as the previous point::
$ docker-compose up -d
Run detectem against some URL::
$ docker-compose run --rm detectem http://domain.tld
The documentation is at ReadTheDocs <https://detectem.readthedocs.io>
_.
.. _Docker: http://docker.io
.. _Splash: https://github.com/scrapinghub/splash
.. _DOM: https://en.wikipedia.org/wiki/Document_Object_Model
.. Docker CE Stable version
: https://www.docker.com/community-edition
.. Docker compose
: https://docs.docker.com/compose/install/
.. _Dockerfile-alternate: extras/docker/Dockerfile-alternate
.. _docker-compose.yml: extras/docker/docker-compose.yml