Pilot regular scrape schedule during agenda posting

[hancush]

We introduced an aggressive scrape schedule on Friday nights (when agendas are posted for events the following week) because we could not rely on the last updated timestamp in the Legistar API to reflect all changes made to board reports and events. However, since that time, we have changed the behavior of windowed scrapes to capture:

• Events: Events updated within the window, as well as all events that have not yet occurred
• Board reports: Board reports updated within the window, as well as all board reports on the agenda of events updated within the window and/or that have not yet occurred

The effect of these changes is that all future events and their agendas are scraped every time we run a windowed scrape. It would be great to simplify the scraping schedule and dramatically reduce the number of calls we make to Legistar.

I propose we pilot using the normal scrape schedule during the next (or, the next convenient) agenda posting Friday. I volunteer to be on call that evening in case things go awry. Thoughts, @camachoo, @shrayshray, and @neilarellano?

[hancush]

We will pilot this ahead of May committee meetings. Agenda posting will happen 5/12, and Hannah will be on call after hours.

[neilarellano]

@hancush did we ever implement this? and if so, what were the results?

[hancush]

@neilarellano no