biglocalnews / civic-scraper

Tools for downloading agendas, minutes and other documents produced by local government
https://civic-scraper.readthedocs.io
Other
40 stars 13 forks source link

Update Legistar to support backfill (and other filter types?) #152

Closed zstumgoren closed 2 years ago

zstumgoren commented 2 years ago

LegistarSite.scrape (and possibly others) does not support the ability to configure downloading of older agendas and minutes, along with other standard behaviors illustrated in CivicPlusSite.scrape. Specifically:

LegistarSite.scrape currently hard codes the date to begin scraping to 2021. We should update the function signature to accept start_date and end_date parameters. These parameters should:

Some additional changes that would be helpful for standardization purposes:

zstumgoren commented 2 years ago

@fgregg I updated the ticket body to more broadly reflect some other possible standardizations as well. Date-based filtering is most important, but it would be great to match the functionalities on CivicPlusSite.scrape as closely as possible