search

j3camero / canada-election-forecast

Code that forecasts the results for each electoral district (riding) in the Canadian 2015 federal election
Apache License 2.0
27 stars 2 forks source link

Fix bad postal code mappings #11

Closed j3camero closed 9 years ago

j3camero commented 9 years ago

Got an email from someone who said their postal code is sending them to the wrong riding. Waiting on them to follow up with a specific postal code so I can reproduce the issue.

j3camero commented 9 years ago

This is confirmed. I've got at least two postal codes from two provinces which map to adjacent ridings. This is potentially quite serious.

I'm withholding the specific postal codes for now for the privacy of those who reported the bug. After a bit of investigation I might be able to find nearby postal codes that also map to the wrong ridings and post those instead.

My first step is to circle back with the nice folks who computed the mapping for me, who for the time being will also remain anonymous for their privacy.

j3camero commented 9 years ago

I've discovered two things: 1) Postal codes for addresses nearby the "defective" postal codes do map correctly. The bad postal codes are one-offs. 2) I've traced the errors back to errors in the postal codes file that was used to compute the mappings.

Taken together, we can rule out the very worst case - that this was some sort of ultra high tech voter suppression scheme on the part of the folks who provided the postal code mappings to me.

It all goes back to errors in the original inputs, which are publicly available: http://geocoder.ca/?freedata=1

This means we have to fix the problem by finding a more accurate source for postal code centroids.

j3camero commented 9 years ago

I've temporarily fixed the two specific postal code manually in the database. The plan is to fix the problem in general by having the website hit this elections canada form in the background:

http://www.elections.ca/scripts/vis/FindED?L=e&PAGEID=20

The responses would be cached in the database, so the site would only hit elections canada a finite number of times. This appears to be within their usage guidelines, but I'll email them before the feature goes live just in case.

j3camero commented 9 years ago

Done https://github.com/j3camero/canada-election-forecast/commit/687f421890ac6769cf9e8182858e57d432f6a84a

The postal codes are now looked up using the following sources, each failing over to the next: 1) Database of previously queried postal codes. This is a caching layer. 2) Elections Canada lookup web form http://www.elections.ca/scripts/vis/FindED?L=e&PAGEID=20 3) Previously used postal code mapping, which contains the occasional inaccuracy