Add filtering step; create site-specific filters

[titaniumbones]

• Add a dictionary or similar data structure that matches each subdomain with a set of filters. Maybe this goes in its own file to avoid confusion
• write a README giving specs for filters; each filter should probably be its own file??
• Add a diffing function that manually compares the retrieved versions
  • ideally, this function will be aware of the DOM (using nokogiri? I'm not a ruby programmer!).
  • iterate through individual diffs; if there is no diff that passes all filter tests, either remove from list (i.e., DO NOT RECORD IN CSV), or give a low priority. @abergman @trinberg @maya you guys should weigh in on which is preferable.

[titaniumbones]

@atesgoral @jpmckinney @sonalranjit anyone else want to take a look at this?

[titaniumbones]

Looks like there is a standard diff library in Ruby: https://github.com/samg/diffy

[atesgoral]

For DOM-aware diffing: https://github.com/postmodern/nokogiri-diff