Some reports are split up over multiple web pages, and we're only fetching the table of contents thus far. For example, http://oig.federalreserve.gov/reports/board-full-report-20140312a.htm points to an executive summary and five sub-pages. This will require multiple URLs and files per report, or maybe crawling the pages we need and stuffing them in a WARC archive, for example. Perhaps scrapers will have to provide a list of URLs for such reports, rather than a single URL.
Some reports are split up over multiple web pages, and we're only fetching the table of contents thus far. For example, http://oig.federalreserve.gov/reports/board-full-report-20140312a.htm points to an executive summary and five sub-pages. This will require multiple URLs and files per report, or maybe crawling the pages we need and stuffing them in a WARC archive, for example. Perhaps scrapers will have to provide a list of URLs for such reports, rather than a single URL.