Handling future councils

[ajparsons]

Currently this dataset only knows about current legal councils. Even though some councils will merge in April 2022, this dataset doesn't know about that, or about the future councils that replace them.

Given we want to be ready to deploy updated datasets based on these councils relatively quickly, and could do so in advance, knowing about future developments would be useful. Democracy Club also have good uses for IDs well in advance of a council legally existing.

Originally this is because the unique ID was dependent on the registers project (rip). Now we're effectively making up our own unique IDs, we could include more information about current council end dates and replacements.

The practical problem with this is some of our scripts assume that a council having a non-null end date means it is a former council (as all end dates are in the past because of how this was updated).

There are two approaches to that problem:

• Make all lookups smarter and check against the current date.
• Produce several versions of the dataset and exclude future knowledge from the 'current' version of the spreadsheet.

The second seems like the easier approach. This would involve an amend to build.py to produce the current data as a '_future' dataset, and producing an amended version that:

• removes any councils with a start-date in the future
• blank out end-date and replaced-by when an end-date is in the future.
• an additional trigger for the build github action of a daily action to update the file when a significant date passes

[ajparsons]

Addressed through splitting into two packages through the new system.