change robots.txt to exclude BLEXBot

[matthewcornell]

It is crawling forecast downloads. I wrote them but heard nothing back. http://webmeup-crawler.com/

[matthewcornell]

Update: BLEXBot is not honoring the new directive:

User-agent: BLEXBot
Disallow: /

Next step: options:

• block at Heroku/router level, say by IP address or user agent
• block at Django middleware level, ""

Of course IP addresses can be forged, as can user agent.