[READY] Re-scrape console command

[andehr]

• ACLEDTagger now defines class DomTaggerOpenAccess which is a DOMTagger with public access to its tagging method tag() which also requires only a String of html. You can acquire an instance of the tagger like this: scraperService.getScraper(source, scraperDir).
• The DateTimeService now provides a parseDate(scrapedDate, source) function which re-parses a scraped date given a Source config. The service also has a simple isInRange(articleDate, from, to).
• The ShellRunner has a new console command "re-scrape", which finds all articles for a specific Source, and using their raw html field, will re-scrape and update their scraped fields.
  • Usage: re-scrape -s "MiMorelia" -sd /path/to/scrapers
  • It will re-parse the date (Article.DATE) if a scrape occurred.
  • You can limit the scope of the re-scrape with a from and to date (using -f and -t). The article's parsed date is used for this boundary check. If the article doesn't have one, then it's included in the re-scrape anyway.
• The ShellRunner function looks a bit big, but is mostly knitting and reporting, leaving the scraperService and datetimeService to have responsibility for individual things. But there's maybe scope for making a scraperService function to house the re-scrape and re-date-parse if you prefer @simonwibberley ?

[simonwibberley]

@andehr could you put implementations in a service? I'm trying to keep the ShellRunner class clean, it just marshals arguments to implementations and shouldn't have any business logic in it...

[andehr]

Sure, I wasn't sure how much of the logic of quite a specific command should be put in general services. So maybe just add a rescrape function to the scraperService which delegates to the datetimeService whenever needed?

[simonwibberley]

@andehr Would have thought the scraper service would be a good place...