Improve documentation - Githubissues

Improve documentation and FAQ's/troubleshooting for the following:

[x] How to yield requests from your spider, this probably needs to go in the Wandering Spider example
[x] Explain how different spiders can talk to each other using the request.meta['spiderid'] field in their request. This is an undocumented feature but works.
[x] Update docs for new kafka consumer and producer settings
[x] Update docs for close() plugin functionality in redis monitor
[X] Update documentation for running both offline and online unit tests (since we switched to nose for offline unit tests)
[x] Ensure changelog is up to date
[x] Update migration docs when 1.2 is completed
[x] Add redis keys docs for actions conducted that are not scrapes
[x] Update documentation about production scheduler domain ttls
[x] Update production deployment documentation about proper Stats configurations (remove 1 week, 24 hrs)
[x] Add compatibility for VM provisioning with centos 7 as well as ubuntu
[x] Add Kafka monitor distributed documentation and diagrams
[x] Add Redis monitor distributed documentation and diagrams
[x] Finalize docker documentation
[x] Correct requirements.txt usage in Overview and Quickstart
[x] Update spider documentation with new middlewares for "how to create your own spider", include what things someone needs to do if they don't want to use the RedisSpider base class
[x] Add Arachnado to comparisons for other distributed scrapy projects

istresearch / scrapy-cluster