For mapping la to oa for scotland, we currently load a 400Mb postcode lookup for the whole of the UK - only to use a tiny bit of rows / columns for the join.
There is another dataset from the national records of scotland, which provides a mapping between local authorities (or commonly called council areas in scotland) to output areas (OA_to_higher_areas.csv).
Research whether we can replace the original datasource. Do a groupBy for each after the join to see if the totals + LA / council area names differ.
For mapping la to oa for scotland, we currently load a 400Mb postcode lookup for the whole of the UK - only to use a tiny bit of rows / columns for the join.
There is another dataset from the national records of scotland, which provides a mapping between local authorities (or commonly called council areas in scotland) to output areas (
OA_to_higher_areas.csv).Research whether we can replace the original datasource. Do a groupBy for each after the join to see if the totals + LA / council area names differ.