Schema for data on election candidates and results - Challenge

Title

Schema for data on election candidates and results

Challenge Owner

Local Government Association - Tim Adams

Short Description

A schema defining the structure data describing an election, the candidates and results. Initially the structure should define a CSV spreadsheet format so that data can be easily published and consumed by non-programmers as well as data experts. The data format should be suitable for first-past-the-post elections in which one or more candidates can be elected for an area.

User Need

The schema is needed particularly for local government elections for which candidates are declared and results published by a few hundred local authorities in different formats.

Although there is no statutory requirement to do so, local authorities generally publish local and national election results on their web sites once those results have been provided to them by the relevant returning officer. There is no guidance or common practice to publish such data in any particular style, format or web location other than the statutory requirement placed on the returning officer to give public notice of the name(s) of the elected candidate(s) (and the fact that they were duly elected), the total number of votes given to each candidate in a contested election and details of the rejected ballot papers as shown in the statement of rejected ballot papers.

Whilst this approach allows scrutiny and review by individuals who discover the local published web pages, the work to locate such information automatically on a larger scale and then to collate data from every local authority to create a national overview is difficult, labour intensive, time consuming and often error prone. Substantial savings and ease of data discovery and reuse is possible if electoral administration departments can be encouraged to publish their data to a simple consistent form which can be read by humans and machines. In May 2016, the Government published its revised National Action Plan for Open Government 2016-2018 and Commitment No. 7 proposes a move towards consistent publishing of elections data to facilitate improved citizen engagement, take-up and innovative re-use by analysts and app developers.

Expected Benefits

Wider outreach of candidates and results data in a timely manner by media outlets, consultancy and advisory networks, political parties and organisations and wider general take-up of the democratic process by data consumers and public and private app developers
Improved analysis of results across different areas
Reduced time and costs associated with comparing and analysing election results

Functional Needs

A schema that defines in human and machine readable format the structure of data describing an election including:

Statements of persons nominated in advance of the election
Election results with votes for each candidate, turnout and who was elected

It is necessary to be able to validate that elections data published:

is in the format given by the schema
uses consistent identifiers for such things as elected bodies, areas represented and political parties such that multiple election datasets can be combines with meaningful results.

The CSV schema needs to be documented in a way that unambiguously allows data experts to extract spreadsheet data into a structured database format.

Work Completed To Date

The LGA has consulted stakeholders and developed a draft standard according to the iStandUK process for standards development. These are the reference documents:

Process For Standards Adoption (rev2) from iStandUK
The latest version of the schema specification: http://e-sd.org/vgTJ3
The launch news item that has guided information about approach: http://about.esd.org.uk/news/towards-common-standard-publishing-consistent-election-results
The Requirements Spec: http://e-sd.org/Yvkkb
The Landscape Spec: http://e-sd.org/U7AYg
The Approach Spec: http://e-sd.org/78z7Q
The summary of consultation 1: http://e-sd.org/fmcAY
The summary of consultation 2: http://e-sd.org/Rsr9V
The minutes of the EMS suppliers meeting: http://e-sd.org/jIfQR
The minutes of the Electoral Commission meeting: http://e-sd.org/2sGSu
Key submission 1 from Assoc of Electoral Administrators: http://e-sd.org/WyVdu
Key submission 2 from Assoc of electoral Administrators: http://e-sd.org/o2BjJ
The online Knowledge Group thread – (people have to register and sign in but its free): https://khub.net/group/localopendatacommunity/forum/-/message_boards/category/18957717
Sample elections data (manually modified) May 2016 local elections: http://e-sd.org/ygEKS
Open elections data validator tool: http://validator.opendata.esd.org.uk/electionresults
Open data URI search tool: http://uris.opendata.esd.org.uk/

On hold due to General Election. We'll use the data generated by this election to understand user needs.

Parliament have published research briefings on the election.

Their full results CSV is available at http://researchbriefings.files.parliament.uk/documents/CBP-7979/hocl-ge2017-results-full.csv

Column headings are

ons_id,ons_region_id,constituency_name,county_name,region_name,country_name,constituency_type,party_name,party_abbreviation,firstname,surname,gender,sitting_mp,former_mp,votes,share,change

It would be useful to get someone from Parliamentary Data Service to see whether your proposed standard meets their needs.

Thanks Terrence. Some columns' contents align with those resulting from LGA consultation. Use of GSS codes (which they call ons_id) aligns with the recommendation to use URIs for GSS codes resolving to statistics.data.gov.uk.

I'll bring this to the attention of the LGA.

Terence. We have consulted with the Parliamentary Data Service and factored the majority of their suggestions into the standard. I have their inputs on file

Tim

Hello. My name is Howard Goodman, and I met Emma Beer at the Civil Service Live conference in Edinburgh on 4 July 2017, and had a brief chat about election data. In my previous job, I worked as a senior developer at the Press Association, and had responsibility for all election systems. We devised our own XML formats for the various types of elections (this was in the early 2000s, when XML reigned supreme and JSON was barely known). You can find all the schemas and examples of their use at http://election.pressassociation.com.

The Press Association has the responsibility of sending all the nominations and results to all the print media and much of the broadcast media. Getting the nominations is always a huge pain, because every council publishes them differently. If a standard format could be agreed upon and used by all councils, then the nominations could be sent directly to PA, saving a huge amount of effort.

There is also an international standards body, iptc.org, which tries to standardise data formats between media organisations around the world. They have a format called SportsML, for instance, but I don’t think there is an ElectionsML.

I have not worked for PA since June 2013, so I’m a bit out of touch, and I don’t know who deals with elections software there now, but if you send a message via http://election.pressassociation.com, it will be picked up by the Elections Editor (currently Roger Smith) and I’m sure he’ll be happy to put you in touch with a developer or technical architect, especially if you say that I sent you!

Thanks for your comments and the information you provided Howard.

In fact, I am aware of this information and these formats you have developed. I have been in conversation with Roger Smith at the Press Association and was collecting his views, ideas and inputs into the standard earlier this year - in March, in fact. Roger also fed inputs to the consultations.

The objective for this standard is to keep it as simple and easily created from the various council run electoral management systems (EMS) as possible. We are doing this for speed, easy validation and versatility for wider re-construction and wider re-use. We agreed that it will be an easy task (much more easy than currently with no consistent form of data publishing from each electoral department) to re-style the basic data into the xml form that the Press Association still uses and relies upon.

We have two other clear markers that we have to accommodate into this work. Firstly, the general convention in local government open data publishing has been to insist to provide data (as a base minimum) as csv data. There are routines and tools that then allow the csv data to be discovered, aggregated and reformatted into json, xml, etc for those that need it.

Second, we are very intent on following the Cabinet Office directed guidance for open public sector data to make maximum use of easy keys within the published data to support linking, reassurance of the data's meaning and consistent connection to other sources through URIs - uniform resource identifiers. I refer to unique references for each council, elected body, ward, local government service, and so on.

One really positive point that Roger Smith made was that the use of csv data could create anomalies in fields that also contain commas within their actual data content. This is very evident in many ward names for example. Roger suggested that we consider tsv - tab separated value - files instead of csv. I am still really awaiting direction on what to do about this. All other local government schemas gave been designed to use csv with quotation marks around data content that includes legitimate commans within them.

Thanks again for you useful and helpful information which has been noted.

Tim

This is a slightly belated reply for me, but thanks very much for yours, and I’m delighted to see you’re already working with PA. It sounds like you have all bases covered, and it seems like quite an exciting project. In a way it’s a pity that I’m no longer involved with it ;-)

There is a similar project for US Election data http://docs.openelections.net/

We should talk to @openelections to see if they have any advice for us.

As part of our evaluation of the election data schema, these are the assessment questions and answers written by the open standards team with Tim Adams .

Applicability

Q1. Does the formal specification address and facilitate interoperability between public administrations?

Yes The schema for election candidates and results defines a format that will facilitate collecting transferring and publishing election data in a common format across public administrations.

Q2. Does the formal specification address and facilitate the development of information or IT systems in government?

Yes By standardisation of election data. Making the development of systems to collect and distribute election information less complex.

Q3. Are the functional and non-functional requirements for the use and implementation of the formal specification appropriately detailed?

Yes The schema has relatively few requirements and these are documented in the schema specification - http://e-sd.org/vgTJ3

Q4. Is the formal specification applicable and extensible for implementations in different domains?

Yes The standard is suitable for use with data from a variety of (first past the post) election types and has the capability to be extended for use in other types if required.

Q5. Is the formal specification largely independent from products of single providers (either open source or proprietary)?

Yes A data standard not tied to any product.

Q6. Is the formal specification largely independent from specific platforms?

Yes The standard is platform independent. Though the schema is a CSV tabular data format.

Q7. Has the standard been written such that its realisation can be/is demonstrated using more than one technology (e.g. XML and JSON)?

No This is a schema for tabular data (CSV)

Maturity

Q8. Has the formal specification been sufficiently developed and in existence for a period to overcome most of its initial problems?

No This is a new schema still under development. Our approval will encourage use and allow it to be tested and improved.

Q9. Are there existing or planned mechanisms to assess conformity of the implementations of the formal specification (e.g. conformity tests, certifications, plugfests etc)?

Yes There is a validator for conformity testing http://validator.opendata.esd.org.uk/electionresults

Q10. Has the formal specification sufficient detail, consistency and completeness for the use and development of products?

Yes The guidelines contain enough detail of the schema and its use. http://e-sd.org/vgTJ3

Q11. Does the formal specification provide available implementation guidelines and documentation for the implementation of products?

Yes The formal specification includes guidelines http://e-sd.org/vgTJ3

Q12. Does the formal specification provide a reference (or open source) implementation?

Not Applicable A simple data schema may not require a reference implementation. The LGA supply validation tools and free to use datasets

Q13. Does the formal specification address backward compatibility with previous versions?

Not applicable No previous versions of this standard exist hence no need for backward compatibility. It is planned future versions of the schema will include compatibility with this version.

Q14. Have the underlying technologies for implementing the formal specification been proven, stable and clearly defined?

Yes CSV, Unicode text, URI, ISO 8601 etc

Openness

Q15. Is information on the terms and policies for the establishment and operation of the standardisation organisation publicly available?

Yes This is an LGA initiative following an iStandUK process for development and adoption of standards http://e-sd.org/OCjay

Q16. Is participation in the creation process of the formal specification open to all relevant stakeholders (e.g. organisations, companies or individuals)?

Yes consultation process open to any interested parties http://about.esd.org.uk/news/towards-common-standard-publishing-consistent-election-results

Q17. Is information on the standardisation process publicly available?

Yes See text of challenge for links

Q18. Information on the decision-making process for approving formal specifications is publicly available?

Yes Information on process published in challenge text on open standards GitHub https://github.com/alphagov/open-standards/issues/42 See - “developed a draft standard according to the iStandUK process for standards development”

Q19. Are the formal specifications approved in a decision-making process which aims at reaching consensus?

Yes The schema specification was developed through consultation with stakeholders. http://e-sd.org/Rsr9V

Q20. Are the formal specifications reviewed using a formal review process with all relevant external stakeholders (e.g. public consultation)?

Yes http://about.esd.org.uk/news/towards-common-standard-publishing-consistent-election-results See consultation document links

Q21. All relevant stakeholders can formally appeal or raise objections to the development and approval of formal specifications?

Yes During public consultations

Q22. Relevant documentation of the development and approval process of formal specifications is publicly available (e.g. preliminary results, committee meeting notes)?

Yes Process is outlined in this document
http://e-sd.org/78z7Q

Q23. Is the documentation of the formal specification publicly available for implementation and use at zero or low cost?

Yes The draft documentation is freely available the schema template will be available (when further matured) as a standardised schema which can be downloaded from http://schemas.opendata.esd.org.uk/

Intellectual property rights

Q24. Is the documentation of the IPR for formal specifications publicly available (is there a clear and complete set of licence terms)?

Yes All LGA developed schemas and supporting documentation produced will be published with an open licence and free to use as defined by https://www.gov.uk/service-manual/making-software/open-standards-and-licensing.html

Q25. Is the formal specification licensed on a royalty-free basis?

Yes The standards and supporting documentation produced will be published with an open licence as defined by https://www.gov.uk/service-manual/making-software/open-standards-and-licensing.html

Market support

Q26. Has the formal specification been used for different implementations by different vendors/suppliers?

No Not yet in use by vendors

Q27. Has the formal specification been used in different industries, business sectors or functions?

N/A Is a new tabular data schema for election information

Q28. Has interoperability been demonstrated across different implementations by different vendors/suppliers?

No This standard has been developed with the aim of increasing take up by vendors and users by creating a simple data schema that can be built on as required.

Q29. Do the products that implement the formal specification have a significant market share of adoption?

N/A This data standard is not intended for use in particular products. The are a number of election management systems that could use this schema and would be encouraged to do so by its selection through this process.

Q30. Do the products that implement the formal specification target a broad spectrum of end-uses?

N/A Data standard is not intended for use in particular products. The schema is intended to increase the availability of election data to end users.

Q31. Has the formal specification a strong support from different interest groups?

Yes List of the interested stakeholder is contained in the guide - http://e-sd.org/vgTJ3

Potential

Q32. Is there evidence that the adoption of the formal specification supports improving efficiency and effectiveness of organisational process?

No None yet - the expected benefits are

Wider outreach of candidates and results
Improved analysis of results across different areas
Reduced time and costs associated with comparing and analysing election results

Q33. Is there evidence that the adoption of the formal specification makes it easier to migrate between different solutions from different providers?

No However, the data schema should improve the reuse of data in different systems

Q34. Is there evidence that the adoption of the formal specification positively impacts the environment?

N/A but a possible reduction in paper forms

Q35. Is there evidence that the adoption of the formal specification positively impacts financial costs?

No One of the rationals of the schema is to reduce time and costs in comparing data. No evidence yet.

Q36. Is there evidence that the adoption of the formal specification positively impacts security?

Q37. Is there evidence that the adoption of the formal specification can be implemented alongside enterprise security technologies?

Q38. Is there evidence that the adoption of the formal specification positively impacts privacy?

Q39. Is the formal specification largely compatible with related (not alternative) formal specifications in the same area of application?

N/A

Q40. Is there evidence that the adoption of the formal specification positively impacts the accessibility and inclusion?

It is possible that tools built to take advantage of the schema could be used to better inform electorate and increase inclusion in the voting process.

Q41. Does the formal specification have a defined maintenance organisation?

Yes The standard has been developed by the Local Government Association

Q42. Does the maintenance organisation for the formal specification have sufficient finances and resources to be sure of freedom from short- to medium-term threats?

Yes The LGA currently has budgeted and contracted technical suppliers to maintain the hosting and oversight of this standard in the future if needs be.

Q43. Does the maintenance organisation have a public statement on intention to transfer responsibility for maintenance of the formal specification if the organisation were no longer able to continue?

Yes The LGA will happily hand over ownership and maintenance responsibility to another party if requested.

Q44. Does the formal specification have a defined maintenance and support process?

Yes Published by LGA and defined in the iStandUK document - http://e-sd.org/OCjay

Q45. Does the formal specification have a defined policy for version management?

Yes All schema templates listed on http://schemas.opendata.esd.org.uk/ have versions listed.

Coherence

Q46. Is this an existing European standard or an identified technical specification in Europe? (Note: CEN, CENELEC and ETSI are the European standards bodies. Technical specifications provided by organisations other than CEN, CENELEC or ETSI can be under consideration to become a European standard or an identified technical specification in Europe for example through the Multi Stakeholder Platform.)

Q47. Does this specification or standard cover an area different from those already identified or currently under consideration as an identified European standard or specification?

Yes

I will provide some more in-depth feedback, but for now: have Democracy Club been consulted? They have definitely a good view of problem/solutions and have been funded by the Electoral Commission to do data publishing work.

@puntofisso yes, they have. I understand that they've worked with the LGA on some of this.

One observation to the proposed schema is about future-proofing. Specifically, the ability to run queries about candidates in a way that the concept of candidate as person is concerned. I think a field for a uniqueCandidateURI should be added.

Let me illustrate this with an example. Candidates sometimes stand in two constituencies. Take Jacob Rees-Mogg. As a researcher, I might want to list all the constituencies in which he's stood. How can I make sure that the Jacob Rees-Mogg who contested Central Fife in 1997 is the same Jacob Rees-Mogg who contested the Wrekin in 2001 and North East Somerset since 2010?

Of course, I'm fully aware that there is no operational support for this level of identification at this stage. But it wouldn't be a costly addition, and it would ensure future-proofing and adoption of the standard outside the UK if appropriate.

Hi Guiseppe

Thanks for the suggestion which is a good longer term vision.

We did consider a means of uniquely referencing elections candidates - possibly through a URI unique to each individual - but (after consultation) rejected this extension at this early stage.

The reasons are several.

Firstly, this is a standard for many election types, not just Parliamentary and some elections have huge numbers of national constituencies such as local authority wards, parishes, etc. The onus on maintaining the many tens of thousands of unique councillor candidates over the years is substantial and we identified no organisation currently prepared or able to develop, own and maintain such a list and a uri resources site.

Second, one of our agreed early parameters to try and move something forward was to not expect local electoral administration departments and returning officers to collect, validate, process or publish any candidate data than they already undertake. There is no funding for this work and we are trying to move forward in stages, starting as simply and achievable as possible and then building on capability if and when this gains traction.

Thirdly, as elections are run by many different organisations it will need central coordination and we are hampered by no unique person identification framework in the country. In time, this will change. We might consider using VERIFY.GOV or NI No or such like in the future.

I will add to the wish list of next steps if we manage to roll out this early first stage (which started in late 2015 and has taken us this long to reach even this stage).

Best wishes

Tim

First off, sorry for not replying earlier. Various factors this year have meant I've constantly been bumping this reply down my todo list. I think there's value in having a public record of these comments even though the format is progressing.

Second, general feedback on the presentation of the spec. It’s quite hard in the above to get to the actual spec in its simple form. It should be simple enough to paste into the first post in this issue, as least as an example, with a short description.

That is, I don’t need the spec document to define what a boolean is before I get to the spec – this has made it a little hard to review for me, and to have confidence that I’m looking at the latest version.

In general, I like that it’s CSV and (in theory) editable by a human. I’ve talked before about this being limited to First Past The Post – great for an MVP and to get moving, but I worry that CSV might not work at all for non-First Past The Post while retaining its human readability and writability. I think it would be useful to explore this early on, as it would help with evaluating the simpler form.

URIs

As I suggested in the minutes to one of the meetings above, I would like to see “curies” rather than URIs in the spec, or just agreed IDs. I think a URI is harder to read, but also I don’t think we should introduce URIs for wards and organisations in this way. There are a few URIs for both of these, and I expect no everyone will agree on the best one for a given use.

This problem goes away if we move to existing, well established IDs for each thing and then leave re-users to decide how they want to link the data up. Using standard IDs doesn’t stop anyone from creating a URI to wherever they want in future (of course this depends on maps between IDs, but these are fairly well maintained at the moment, and don’t create any form of lock in).

On ElectedBodyURI, I don’t understand why, in the sample file, its a URI to Ordnance Survey, or why the URI points to their MetropolitanDistrict type. Firstly OS isn’t an authority on local authority types, but I also don’t understand what value is added in that column – the organisation is already identified, and the type should be knowable through that. Taking the design principle that data should be minimal, I think this column isn’t adding anything.

I’m fairly sure the Electoral Commission URIs will change in the future too, but the party IDs are more stable.

Open Data Communities is England only

Unless I’m mistaken, this will only work for English local authorities if the URI is prescribed to be open data communities.

As above, I would like to see IDs used rather than URIs, and I think the IDs in the various registers are going to be the best – although some are still in alpha.

GSS codes can’t work for wards

GSS codes can’t be used for wards when there are boundary changes. We’ve written a lot about this, but in short, the codes for new areas aren’t made public before an election happens, or even directly after. This means there won’t be a URI or ID for a new ward after the election happens. This means ElectoralAreaURI can’t be populated.

This is because OS won’t routinely open data on boundaries before an election happens, ONS don’t assign IDs to the data the boundary commission creates, and The Boundary Commission(s) don’t issue stable IDs for areas.

This mess means that, as things stand, there won’t be a GSS code for any newly created area before the election is held.

Of course, this only applies to new areas, but as every area is reviewed every few years, on going this means that it applies to everywhere at some point.

Democracy Club have fudged this (and it is a fudge) by asserting that the organisation ID and the slug of the ward name as defined in legislation will always be unique at any one time. That is, ‘St Helen’s’ in West Oxfordshire (a made up place as far as I know) will have an ID of WOX:st-helens, and that ID will be unique for that set of boundaries (in other words, unique for all wards that have the same start and end date).

This is far from ideal, but it’s about all we have.

The ideal would be for The Boundary Commission(s) to create IDs for each geography at the point they create them. That is, at the point the first shapes are drawn at the start of the consultation process.

The Boundary Commission(s) are, after all, the legal body in charge of the boundaries, so they should have the authority and duty to issue IDs for them.

This is slightly out of scope of this issue though.

Of course, we can use GSS codes where they are published. Of course, GSS codes change for minor boundary changes that don’t trigger elections, so an ID for the ward that’s stable between Electoral Change Orders might be valuable too. I think this is another reason to use a curie rather than a URI.

Extra info

There is a fair amount of additional info, like the InfoURL, ReturningOfficer and ContactEmail. This is of course useful, but I think shouldn’t be required. Will ElectoralDept always be “Electoral Services”? If so, can it be removed? What value does it add?

On ServiceTypeURI andServiceTypeLabel – will this always be the same (“Election results “)? What value does it add to this format?

What are the enumerations of TypeofElection? Is (ElectionDate, TypeofElection) always unique?

Emblems

Are emblems desired? They make up the ballot paper, and I think might be useful for people, but I know that there aren’t reliable IDs for emblems yet, so this might need to be added at a later date.

Hi Sym

Thanks for the lengthy feedback. It is a little late as the consultation is closed and we have now moved forward with the specification to work towards pilot implementations in coming weeks. However, you make some very good points band observations. I have tried to respond by taking your narrative above and then inserting my responses as embolden italic fonts. They are also red in my local version but I am not sure if the colours carry over to GitHub.

You started with…. First off, sorry for not replying earlier. Various factors this year have meant I've constantly been bumping this reply down my to do list. I think there's value in having a public record of these comments even though the format is progressing.

The consultation is now closed but we can keep your comments as noted for refinements of future versions after we have monitored the successes or failure of early implementation. Thanks for sending this thorough review but it has come a little late. We have run three consultations on this project starting back in March 2016 and finishing in October 2017. This is the first thorough response from your organisation, though we did take on board consideration of the points and ideas that you raised in our collective meeting with EMS suppliers in July 2016. Our response and conclusions to many of your points and worries have valid underlying reasons why they have turned out in the way that they have. We have had to accommodate the views and interests of quite a large cross section of different stakeholders. We were necessarily guided by our board to keep things simple at this early stage of development to engender confidence, buy-in and strong chance of early wins. We are also undertaking this work under the standards development frameworks that are in operation in the local government sector that are guided by Cabinet Office and the Local eGovernment Standards Body. The proposed standard has also been strongly reviewed and scrutinized by the Cabinet Office Open Standards Board throughout most of 2017 and they have endorsed the standard as a first approach from which to build further.

Thanks for the comment. We have adopted the same format and content/style that has been adopted for all local government centrally coordinated data standard specifications since 2014. We will look to improve the layout and approach of future versions. The detailed table in the Appendix covers everything that technical experts like yourself needs to read. However, the document has had to accommodate the interest, involvement and understanding of a wide range of different levels of data and governance skills, capability and expertise including returning officers, librarians, senior managers in public sector organisations as well as data analysts, system developers, etc.

That is, I don’t need the spec document to define what a Boolean is before I get to the spec – this has made it a little hard to review for me, and to have confidence that I’m looking at the latest version.

The subject and data area is complex. The strategy has always been to start simply and in a non-threatening or challenging way to generate interest and early results. If there is good take-up and successful data re-use we can look to adopt a more advanced approach once early benefits have been demonstrated In local government we are looking at online free-to-use tools that allow conversion of data from one form to another such as csv to xml to Json, etc. Ideally we need to define the underlying data model and output schemas, which may need to be in a richer format than CSV if/when we progress to providing more information and supporting FPTP. .

URIs As I suggested in the minutes to one of the meetings above, I would like to see “curies” rather than URIs in the spec, or just agreed IDs. I think a URI is harder to read, but also I don’t think we should introduce URIs for wards and organisations in this way. There are a few URIs for both of these, and I expect no everyone will agree on the best one for a given use.

We considered using CURRIEs and breaking down URIs into three or four constituent parts during earlier discussions in the consultation on GitHub. We discarded doing so in this first version. We have adopted the same approach being used successfully in most of the other centrally coordinated local government data standards to-date. There are five URIs in the elections data standard which would increase the number of columns of data by an additional 15 columns if CURRIEs were used. Moreover, in local government the full URI components are able to be resolvable to their source location (either human readable or machine to machine) so that further data linking and new data constructs can be encouraged. A simple algorithm is able to subdivide a URI into its constituent parts if that is important to your business. Our focus was to try and keep simplicity and flexibility at the centre of the proposal. Use of CURIEs begs the questions as to where they are defined so full URIs can be derived. We need a common approach to this. Options are: to include extra columns with full URI and/or base URI; and have an external file with metadata that resolves CURIEs. To date we have included just a single URI field and the label of the URI for simple human readability. In the absence of consistent government wide approach and guidance we propose sticking with the full URI and label. There is much debate and differing views on this challenge and we are keen to encourage a wider conversation, perhaps led by GDS on government policy for general open data publishing guidelines. The LGA is open to support whatever turns out to be the consensus. The Open Data Institute (ODI) is also currently looking at this in the mix of its development of guidance and policy for wider open data publishing in the public sector. The LGA and our partners is contributing to this initiative.

This problem goes away if we move to existing, well established IDs for each thing and then leave re-users to decide how they want to link the data up. Using standard IDs doesn’t stop anyone from creating a URI to wherever they want in future (of course this depends on maps between IDs, but these are fairly well maintained at the moment, and don’t create any form of lock in).

We are moving towards well established URI sources but these are still battling with each other currently until the more successful sources win over. We can change URI sources as the open data business in the UK matures and settles down in these early years of exploration and innovation. The sources of URIs that we have proposed are well rehearsed and shown stability in Local Government in recent years but we can always learn and change them. While the specification might specify that a URI is needed, the source of the URI can change over time and for different types of election. The advantage of using a full URI is that it can be resolved without reference to any external guidance as to where the URI come from..

On ElectedBodyURI, I don’t understand why, in the sample file, it’s a URI to Ordnance Survey, or why the URI points to their MetropolitanDistrict type. Firstly OS isn’t an authority on local authority types, but I also don’t understand what value is added in that column – the organisation is already identified, and the type should be knowable through that. Taking the design principle that data should be minimal, I think this column isn’t adding anything.

Agreed. Ordnance Survey has a voting area ontology and indicated that they were likely working towards a catalogue of voting bodies hung off this ontology but that seems not to be the case after two years. We are going to drop the OS as a data source and downgrade the cardinality to 0 or 1 for ElectedBodyURI. This will make the label mandatory but the URI optional (and likely always empty until we can identify a source). The LGA might consider developing a Voting Body register to GDS standards for future use but no decisions are yet made.

I’m fairly sure the Electoral Commission URIs will change in the future too, but the party IDs are more stable.

We are in discussion with Electoral Commission to develop more formal persistent URIs from them and the expansion of their API capability for future use. They were ready and supportive in 2016 but we need to restart discussions given the 12 month delay whilst we have undergone GDS scrutiny.

Open Data Communities is England only Unless I’m mistaken, this will only work for English local authorities if the URI is prescribed to be open data communities.

Agreed. We point out this is an English project. The source of the URIs is not mandated so other parts of UK can take part if they wish. In time we will move to GDS registers. However, this is resisted currently because some registers are held at beta status and unlike open data communities they do not provide links to other data sets known about the areas.

As above, I would like to see IDs used rather than URIs, and I think the IDs in the various registers are going to be the best – although some are still in alpha.

See my points made earlier on this subject.

GSS codes can’t work for wards GSS codes can’t be used for wards when there are boundary changes. We’ve written a lot about this, but in short, the codes for new areas aren’t made public before an election happens, or even directly after. This means there won’t be a URI or ID for a new ward after the election happens. This means ElectoralAreaURI can’t be populated.

We think that the spec is clear and takes account of your worries. We are using ONS as a source of statistical geographies down to the level covered by that organisation and we will support lower levels not supported by ONS using the local government natural neighbourhood URI sources. We spend considerable resources and costs in local government maintaining this free open data service. We have a special arrangement with Boundaries Commission to secure new updates the moment they are released – well before an election and before they are published by Ordnance Survey. We also have an arrangement with ONS to provide auto redirects between GSS and natural neighbourhoods when they become necessary to maintain URI persistence.

See above. Local Government does handle these records as open data once they are available from Boundary Commission. In parallel, the LGA continues to pressure the relevant parts of government to release published pre-operative boundaries in the same way as it does operative boundaries.

This mess means that, as things stand, there won’t be a GSS code for any newly created area before the election is held.

Of course, this only applies to new areas, but as every area is reviewed every few years, ongoing this means that it applies to everywhere at some point.

Democracy Club have fudged this (and it is a fudge) by asserting that the organisation ID and the slug of the ward name as defined in legislation will always be unique at any one time. That is, ‘St Helen’s’ in West Oxfordshire (a made up place as far as I know) will have an ID of WOX:st-helens, and that ID will be unique for that set of boundaries (in other words, unique for all wards that have the same start and end date).

This is far from ideal, but it’s about all we have.

The ideal would be for The Boundary Commission(s) to create IDs for each geography at the point they create them. That is, at the point the first shapes are drawn at the start of the consultation process.

The Boundary Commission(s) are, after all, the legal body in charge of the boundaries, so they should have the authority and duty to issue IDs for them.

This is slightly out of scope of this issue though.

Extra info There is a fair amount of additional info, like the InfoURL, ReturningOfficer and ContactEmail. This is of course useful, but I think shouldn’t be required. Will ElectoralDept always be “Electoral Services”? If so, can it be removed? What value does it add?

On ServiceTypeURI andServiceTypeLabel – will this always be the same (“Election results “)? What value does it add to this format?

The standard is accommodating as many views, requests for approach and systems that we have made contact with to link to other data sets in the country. This is why those fields that are not so interesting or useful to you are included. In the grand scheme of data collection and the hundreds of varying and conflicting comments and requests we have received over the last two years, these data links included are easy to incorporate and are relatively normal already in local government. You can ignore them if they have no interest to you.

What are the enumerations of TypeofElection? Is (ElectionDate, TypeofElection) always unique?

Yes and they were added in this way to support the Democracy Club vision of uniquely identifying an election

The enumerations are: Parish, ParishByElection, DistrictAndBorough, DistrictAndBoroughByElection, County, CountyByElection, Parliamentary, ParliamentaryByElection

Emblems Are emblems desired? They make up the ballot paper, and I think might be useful for people, but I know that there aren’t reliable IDs for emblems yet, so this might need to be added at a later date.

Electoral Commission has indicated that they can add emblem jpegs to their forthcoming API extensions but there is some uncertainty on historic versions so we decided to look at this again in later versions

It may be possible in future versions to accommodate emblems associated with a party identifier or with their own unique identifiers.

STD04397 - Democracy Club comments of Election data Std 171110.docx _**

Hi Tim,

I wanted to bring your attention to recent developments around registers in this space, which may be beneficial.

Registers for local authorities in England, Scotland and Wales are now available and ready to use. Each of those links to a register of local authority types which is also ready to use.

There is a linked statistical geography available for Wales, and one open for feedback for Scotland.

A register of local authorities in Northern Ireland has just been opened for feedback - we'd welcome any suggestions from this group on any of our 'open for feedback' data.

I hope this enables this project to utilise the local authority registers, but if there are further obstacles please let me know and I can see how best to support. For example, it would be useful to understand more about the links to other data that you would find beneficial around these registers, mentioned in your response above.

Thanks, Ellie

Product Manager Registers GDS

Thanks Ellie Yes we are aware and have been working with GDS and DCLG to develop them. we had many discussions about their use in the initial elections data open standard with many parties including GDS and the Standards Board and the iStandUK team. At this stage we decided to remain with our existing reference sources and URI sets used widely in local government standards to preserve compatibility for the time being but (most importantly) the registers status alpha, beta, etc have been uncertain for many months and there have been many Cabinet Office initiatives over the decades that seem to come and go,

Our biggest concern, currently, id that the registers do not offer the powerful extra facilities of resolving to a definitions page that includes many supporting links to associated data to encourage app innovation, Linked Data techniques.

I expect that we will move to the registers in due course (perhaps a future version of the standard) once we monitor interest, take-up, etc

Tim

We've reviewed all the feedback on the proposed elections data standard and are delighted to tell you that it meets our definition of an Open Standard suitable for use in government.

Normally our next step would also include Open Standards Board approval - however, as this is mostly a matter for local governments, we wouldn't want to be seen as interfering in those processes. In addition, we are currently recruiting for new board members - so there would be a delay before a new board is convened.

That said, there is now no obstacle from promoting this as an Open Standard. We will be adding it to the list of recommended standards as soon as possible.

We look forward to seeing this implemented across the country.

co-cddo / open-standards