Site.scrape method should be updated to match the features supported in CivicPlus and Legistar. Specifically, we should attempt to implement the following features:
date-based filtering, with default set to current day
file type filtering for downloads, with Agendas and Minutes set to default
file size filtering for downloads
We should also attempt to implement the cache flag as in CivicPlus, although we didn't do so for Legistar since the underlying python-legistar framework would need to be refactored to support caching of source HTML.
Site.scrapemethod should be updated to match the features supported in CivicPlus and Legistar. Specifically, we should attempt to implement the following features:Siteand import asPrimeGovSite(the pre-existing name incivic_scraper.platforms.__init__.py)downloadshould be set toFalseby defaultWe should also attempt to implement the
cacheflag as in CivicPlus, although we didn't do so for Legistar since the underlying python-legistar framework would need to be refactored to support caching of source HTML.