estin / pomp

Screen scraping and web crawling framework
https://pomp.readthedocs.org
Other
60 stars 10 forks source link
asyncio crawler crawling framework python scraping

Pomp

.. image:: https://img.shields.io/circleci/project/github/estin/pomp/master.svg :target: https://circleci.com/gh/estin/pomp/tree/master :alt: circleci

.. image:: https://img.shields.io/codecov/c/github/estin/pomp/master.svg :target: https://codecov.io/gh/estin/pomp/branch/master :alt: codecov

.. image:: https://img.shields.io/pypi/v/pomp.svg :target: https://pypi.python.org/pypi/pomp/ :alt: Latest PyPI version

.. image:: https://img.shields.io/pypi/pyversions/pomp.svg :target: https://pypi.python.org/pypi/pomp :alt: python versions

.. image:: https://img.shields.io/pypi/wheel/pomp.svg :target: https://pypi.python.org/pypi/pomp/ :alt: Have wheel

.. image:: https://img.shields.io/pypi/l/pomp.svg :target: https://pypi.python.org/pypi/pomp/ :alt: License


Pomp is a screen scraping and web crawling framework. Pomp is inspired by and similar to Scrapy, but has a simpler implementation that lacks the hard Twisted dependency.

Features:

Pomp makes no attempt to accomodate:

If you want proxies, redirects, or similar, you may use the excellent requests_ library as the Pomp downloader.

Pomp examples_

Pomp docs_

Pomp is written and maintained by Evgeniy Tatarkin and is licensed under the BSD license.

.. _Scrapy: http://scrapy.org/ .. _Twisted: http://twistedmatrix.com/ .. _concurrent.futures: http://pythonhosted.org/futures/ .. _Pomp examples: https://github.com/estin/pomp/tree/master/examples .. _Pomp docs: http://pomp.readthedocs.org .. _Paste: http://pythonpaste.org/ .. _requests: http://www.python-requests.org/en/latest/