Crawling strategy is moved to FrontierManager

[sibiryakov]

Mainly to ease the local debugging of crawling strategy, to avoid setting up all the dependencies required for distributed version.

• [x] design how this would work when running from SW and using MessageBusBackend,
• [x] extract DomainCache interface, make it part of Backend,
• [x] implement SQLAlchemy implementation for DomainCache,
• [x] instantiate DomainMetadata in HBaseBackend,
• [x] decide on old non-distributed backends,
• [x] remove old, non-distributed backends,
• [x] rewrite single process quick start to use Sqlite as a backend,
• [x] test local SQLA backend,
• [x] document available crawling strategies,
• [x] decide what run modes are available after this change,
• [x] update source code and documentation accordingly.
• [x] Mikhail's suggestions.

[codecov[bot]]

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@da87cbd). Click here to learn what that means. The diff coverage is 45.38%.

@@            Coverage Diff            @@
##             master     #331   +/-   ##
=========================================
  Coverage          ?   59.95%           
=========================================
  Files             ?       78           
  Lines             ?     6136           
  Branches          ?      809           
=========================================
  Hits              ?     3679           
  Misses            ?     2280           
  Partials          ?      177

Impacted Files	Coverage Δ
frontera/core/models.py	94.73% <ø> (ø)
frontera/worker/components/batch_generator.py	57.44% <ø> (ø)
frontera/utils/graphs/data.py	81.81% <ø> (ø)
...ontera/contrib/messagebus/kafka/offsets_fetcher.py	14.89% <ø> (ø)
frontera/utils/twisted_helpers.py	78% <ø> (ø)
frontera/strategy/discovery/sitemap.py	0% <0%> (ø)
frontera/strategy/depth.py	0% <0%> (ø)
frontera/strategy/discovery/__init__.py	0% <0%> (ø)
frontera/worker/components/scoring_consumer.py	72.97% <0%> (ø)
frontera/contrib/backends/remote/messagebus.py	80% <0%> (ø)
... and 24 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update da87cbd...03a5f91. Read the comment docs.

[sibiryakov]

Thanks for review!