search

opencivicdata / python-legistar-scraper

Scrapes municipal data from Legistar websites
BSD 3-Clause "New" or "Revised" License
42 stars 28 forks source link

only count a year as having an event if we yield event, skip duplicate events #91

Closed fgregg closed 5 years ago

fgregg commented 5 years ago

Sometimes there are events that appear in every year we search, so this code handles this.

hancush commented 5 years ago

@fgregg can you say more about why we don't want recurring events? if a board meeting happens annually, i'd want an event for every year. perhaps an example would be illuminating.

fgregg commented 5 years ago

we want all types of events, but only want the scraper to emit each individual event once (where an invididual event is equivalent to an event with a unique GUID)

hancush commented 5 years ago

@fgregg sure, ok. do you have a sense of why the same event shows up in multiple search years?

fgregg commented 5 years ago

That's a deep question that this PR papers over. I'll open up an issue for that. Regardless, I do think that this code is more correct than what it replaces.