Update default_request_filter to exclude more bots

assaf / vanity

Experiment Driven Development for Ruby

http://vanity.labnotes.org

MIT License

1.55k stars 270 forks source link

Closed peterkovacs closed 8 years ago

peterkovacs commented 8 years ago

It will exclude any User-Agent with the string 'bot', 'crawler', or 'spider'.

phillbaker commented 8 years ago

Looks good. Based on http://www.useragentstring.com/pages/Crawlerlist/, this should help with blocking some non humans.