scrapinghub / spidermon

Scrapy Extension for monitoring spiders execution.
https://spidermon.readthedocs.io
BSD 3-Clause "New" or "Revised" License
524 stars 94 forks source link
crawling hacktoberfest monitoring monitoring-tool scraping scrapinghub spiders testing

========= Spidermon

.. image:: https://github.com/scrapinghub/spidermon/workflows/CI/badge.svg?branch=master :target: https://github.com/scrapinghub/spidermon/actions?workflow=CI :alt: CI Status

.. image:: https://img.shields.io/codecov/c/github/scrapinghub/spidermon.svg :target: http://codecov.io/github/scrapinghub/spidermon?branch=master :alt: Coverage report

.. image:: https://img.shields.io/pypi/v/spidermon.svg :target: https://pypi.python.org/pypi/spidermon :alt: pypi version

.. image:: https://img.shields.io/pypi/l/spidermon.svg :target: https://github.com/scrapinghub/spidermon/blob/master/LICENSE :alt: licence

.. image:: https://img.shields.io/pypi/pyversions/spidermon.svg :target: https://pypi.python.org/pypi/spidermon :alt: python versions

.. image:: https://img.shields.io/badge/code%20style-black-000000.svg :target: https://github.com/ambv/black :alt: Code style: black

Overview

Spidermon is an extension for Scrapy spiders. The package provides useful tools for data validation, stats monitoring, and notification messages. This way you leave the monitoring task to Spidermon and just check the reports/notifications.

Requirements

Install

The quick way::

pip install spidermon

For more details see the install section in the documentation: https://spidermon.readthedocs.io/en/latest/installation.html

Documentation

Documentation is available online at https://spidermon.readthedocs.io/ and in the docs directory.