The itc scraper had stopped getting anything at all, without any warnings or errors. I noticed this because the itc directory wasn't there at all in the new archive I made in November, so the scraper had already broken by then without warning.
This rewrites the scraper to handle Audit reports, but not for Semiannual or Peer Reviews. They shouldn't be hard to do. If anyone wants to hop in on this branch and finish the job, I'd much appreciate it.
I've also added a check at the top of the scraper that throws an exception if it finds 0 reports on the audits page. The scrapers should make a point to choke if unexpected behavior happens that suggests a too-graceful handling of changed markup.
The
itcscraper had stopped getting anything at all, without any warnings or errors. I noticed this because theitcdirectory wasn't there at all in the new archive I made in November, so the scraper had already broken by then without warning.This rewrites the scraper to handle Audit reports, but not for Semiannual or Peer Reviews. They shouldn't be hard to do. If anyone wants to hop in on this branch and finish the job, I'd much appreciate it.
I've also added a check at the top of the scraper that throws an exception if it finds 0 reports on the audits page. The scrapers should make a point to choke if unexpected behavior happens that suggests a too-graceful handling of changed markup.