search

unitedstates / inspectors-general

Collecting reports from Inspectors General across the US federal government.
https://sunlightfoundation.com/blog/2014/11/07/opengov-voices-opening-up-government-reports-through-teamwork-and-open-data/
Creative Commons Zero v1.0 Universal
107 stars 21 forks source link

Add Smithsonian #153

Closed spulec closed 10 years ago

spulec commented 10 years ago

The unusual thing about this is that a given report can be seen on multiple of the pages we scrape. On each page, there are different granularities of data. We scrape pages from most granular to least granular and skip report ids that we've already seen. This relies on us being very confident in our report id uniqueness, but I feel pretty good about it. I'm certainly open to better solutions though.

konklone commented 10 years ago

I had glanced at this one and thought it looked very tricky, given the spread out dates. Thanks for all your hard work on it, this looks great, @spulec!

959110_20090407_790screen001