search

DemocracyClub / WhoCanIVoteFor

🗳 The source for https://whocanivotefor.co.uk/
https://whocanivotefor.co.uk/
40 stars 31 forks source link

Normalise postcodes #2044

Open symroe opened 1 month ago

symroe commented 1 month ago

When we get a postcode from a user it can take some different forms:

  1. With or without space
  2. Caps or no caps
  3. Any combination of the above

This is a problem in a few ways:

This causes downstream problems with caching and querying logs. Not having a canonical URL for a postcode might cause undercounts in analytics.

In both of these case we validate that the postcode is valid before showing a URL or logging it, but we don't normalise it.

In other projects we have a Postcode class that can normalise user-provided data into a single form.

We should implement that here (or add that class to DC utils) and convert string postcodes to that early on in the process. We can then use a Postcode class for URLs, logging, etc.

chris48s commented 1 month ago

I know that code :)

There's a Postcode class in the uk-geo-utils package which is mostly the same as this

https://democracyclub.github.io/uk-geo-utils/postcode/ https://github.com/DemocracyClub/uk-geo-utils/blob/4f7b458667679882fd03ddc2a7c104f58b1d89ff/uk_geo_utils/helpers.py#L28-L56

Looking at the code snippet you posted, I'm guessing the reason you chose to copy/paste rather than use that is because uk-geo-utils depends on django (helpers.py even includes imports from django.apps import apps and from django.conf import settings) and you wanted to use that in a starlette app, which makes sense tbf.

I suggest we add the outcode() method to that class and find a way to share that between projects such that Postcode isn't coupled to django. Maybe it moves to a different package, or maybe we could make django an optional dependency of uk-geo-utils (i.e: pip install uk-geo-utils[django]) and move Postcode to a file where it can be imported without having to load django imports? It feels like quite a fundamental thing to be able to usefully share across our projects anyway.