Open mhl opened 7 years ago
A couple of other notes:
Post
has ForeignKey
fields referencing Organization
and Area
, so if the linked Organization
or Area
objects in Django's ORM had been deleted then that would cause the Post
to be deleted as well.LogEntry
for deletion of an Organization
(so it couldn't account for all of these problems) and none for deletion of an Area
. So think I think we can rule out someone deleting these in the admin (although they still might have been deleted in the Django shell, or in the database shell).uk_create_elections_from_every_election
deletes objects ever - just creates or updates them, so I think we can rule out an change in EveryElection causing the posts to be deleted.I've checked the current source code, and I can't see any way by which posts / organizations / areas could end up being deleted.
@symroe and I discussed this today, and decided that the next step should be a regularly run script to detect if any post has been deleted, so we can find out whether this is still happening, and if so localize in time the events that might have caused it. With us both being so busy with the imminent election, this seems like the most important step - the recovery of data can be done from these past local elections can happen after the election.
An update on the posts that have disappeared: because ResultEvent
still is from the PopIt era, it stores post IDs and post names as text, so we can use this find some other posts that have been deleted. These are:
2 Milton Keynes local election no post from: UTW:E05009411 Campbell Park and Old Woughton ward
2 Milton Keynes local election no post from: UTW:E05009424 Woughton and Fishermead ward
2 Portsmouth local election no post from: UTW:E05002447 Eastney and Craneswater ward
2 Southampton local election no post from: UTW:E05002463 Millbrook ward
2 Southend-on-Sea local election no post from: UTW:E05002212 Belfairs ward
2 Southend-on-Sea local election no post from: UTW:E05002213 Blenheim Park ward
2 Southend-on-Sea local election no post from: UTW:E05002214 Chalkwell ward
2 Southend-on-Sea local election no post from: UTW:E05002215 Eastwood Park ward
2 Southend-on-Sea local election no post from: UTW:E05002218 Milton ward
2 Southend-on-Sea local election no post from: UTW:E05002219 Prittlewell ward
2 Southend-on-Sea local election no post from: UTW:E05002220 St Laurence ward
2 Southend-on-Sea local election no post from: UTW:E05002221 St Luke's ward
2 Southend-on-Sea local election no post from: UTW:E05002223 Southchurch ward
2 Southend-on-Sea local election no post from: UTW:E05002224 Thorpe ward
2 Southend-on-Sea local election no post from: UTW:E05002225 Victoria ward
2 Southend-on-Sea local election no post from: UTW:E05002227 West Leigh ward
6 Warrington local election no post from: UTW:E05011028 Chapelford and Old Hall ward
6 Warrington local election no post from: UTW:E05011032 Great Sankey North and Whittle Hall ward
6 Warrington local election no post from: UTW:E05011036 Lymm North and Thelwall ward
4 Warrington local election no post from: UTW:E05011037 Lymm South ward
I've been through the SOPN and I believe these are all the candidacies missing from local.warrington.2016-05-05
:
Chapelford and Old Hall https://candidates.democracyclub.org.uk/person/8902/allan-keith-bird https://candidates.democracyclub.org.uk/person/8903/phil-hayward https://candidates.democracyclub.org.uk/person/8908/faisal-rashid (merge required) https://candidates.democracyclub.org.uk/person/8911/mike-wass https://candidates.democracyclub.org.uk/person/8912/ian-anthony-wilson
Grappenhall https://candidates.democracyclub.org.uk/person/5635/ryan-bate
Great Sankey North & Whittle Hall https://candidates.democracyclub.org.uk/person/450/stephanie-davies https://candidates.democracyclub.org.uk/person/9008/dan-price
Lymm North & Thelwall https://candidates.democracyclub.org.uk/person/5069/bob-barr
Lymm South https://candidates.democracyclub.org.uk/person/854/james-ashington https://candidates.democracyclub.org.uk/person/9351/ed-gough
Warrington has all-out elections in the 2016-2020 cycle, so many of these candidates will be standing again in the postponed election in May 2021.
These are all the candidacies still missing from local.warrington.2016-05-05
https://candidates.democracyclub.org.uk/person/450/stephanie-davies https://candidates.democracyclub.org.uk/person/9008/dan-price https://candidates.democracyclub.org.uk/person/5069/bob-barr https://candidates.democracyclub.org.uk/person/854/james-ashington https://candidates.democracyclub.org.uk/person/9351/ed-gough https://candidates.democracyclub.org.uk/person/7633
Southend-on-Sea local election 2016
@sjorford spotted this worrying case of Jason Pilley's candidacy in a local election disappearing:
https://candidates.democracyclub.org.uk/person/7633
The candidacy was originally added on the 28th of March 2015 as standing in the Southend-on-Sea local election (St Luke's ward) for the Green Party. However, on the 9th of May 2017 an edit to that profile looks as if removed the St Luke's ward candidacy. That doesn't necessarily mean that that was actually done as part of that change - if something had caused the candidacy to be removed without recording a LoggedAction, it would be recorded as part of the next change made to the candidate's profile - so it might have been deleted between 28th of March 2015 and 9th of May 2017.
Investigating further, it looks like there are now no candidates marked as standing in Southend-on-Sea: https://candidates.democracyclub.org.uk/numbers/election/local.southend-on-sea.2016-05-05/posts
... and worse, the election has no posts associated with it:
It seems that these
Post
objects have been genuinely deleted, not just lost their association with the election - I've checked to see if a few posts for that election (from http://www.southend.gov.uk/info/200400/elections_and_registering_to_vote/308/election_results/2 ) exist, but none of those I've checked do. :unamused:We need to work out what might have caused these posts to be deleted - whether it was a mistaken manual intervention, or a bug causing the post to be deleted.
Portsmouth local election 2016
@jf1 spotted this problem which appears to be very similar. He reported:
As with the previous example, it seems that this
Post
in the Portsmouth local election from 2016 has been deleted. Going by:... compared to http://democracy.portsmouth.gov.uk/mgElectionResults.aspx?ID=5&V=1&RPID=502526907
It is the only one from Portsmouth to have been deleted.
Possible other cases
These past local elections have no candidates associated with them, so might also have been subject to the same problem:
(Of course, given the Portsmouth example above, there might be other elections that have had some but not all posts deleted.)