Cache tool presence - Githubissues

unitedstates / inspectors-general

Collecting reports from Inspectors General across the US federal government.

https://sunlightfoundation.com/blog/2014/11/07/opengov-voices-opening-up-government-reports-through-teamwork-and-open-data/

Creative Commons Zero v1.0 Universal

107 stars 21 forks source link

Cache tool presence #260

Closed divergentdave closed 8 years ago

divergentdave commented 8 years ago

This cuts the number of processes spawned in half by memoizing whether each conversion tool is present. In unscientific testing with one scraper, this cut a five to seven second runtime down to four seconds.

konklone commented 8 years ago

Great idea!