Block IP from crawlers

[gtalarico]

Server is getting slammed with crawlers trying to find stuff + 404 errors Gunicorn Server Hooks http://stackoverflow.com/questions/40951861/how-to-use-variables-created-in-gunicorns-server-hooks

Gunicorn Docs http://docs.gunicorn.org/en/stable/settings.html#server-hooks

Crawler Blacklist: https://myip.ms/browse/blacklist/Blacklist_IP_Blacklist_IP_Addresses_Live_Database_Real-time

Add custom rule ?if crawler to hit /blog/ block ip

[gtalarico]

http://esd.io/blog/flask-apps-heroku-real-ip-spoofing.html