Error due to WPC areas in May 2022 boundary line data

[torotil]

When doing a fresh install with the May 2022 boundary line data I get an error during the import:

  New area: WPC W09000029 41405 Cardiff West P Const
Traceback (most recent call last):
  File "/home/roman/code/mapit-build/mapit/mapit_gb/management/commands/mapit_UK_import_boundary_line.py", line 114, in handle_label
    m = Area.objects.get(codes__type=code_version, codes__code=ons_code)
  File "/home/roman/code/mapit-build/.venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/roman/code/mapit-build/.venv/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
    raise self.model.DoesNotExist(
mapit.models.Area.DoesNotExist: Area matching query does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/roman/code/mapit-build/mapit/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/roman/code/mapit-build/.venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/home/roman/code/mapit-build/.venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/roman/code/mapit-build/.venv/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/roman/code/mapit-build/.venv/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/home/roman/code/mapit-build/.venv/lib/python3.9/site-packages/django/core/management/base.py", line 571, in handle
    label_output = self.handle_label(label, **options)
  File "/home/roman/code/mapit-build/mapit/mapit_gb/management/commands/mapit_UK_import_boundary_line.py", line 129, in handle_label
    type=Type.objects.get(code=area_code),
  File "/home/roman/code/mapit-build/.venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/roman/code/mapit-build/.venv/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
    raise self.model.DoesNotExist(
mapit.models.Type.DoesNotExist: Type matching query does not exist.

It seems there is no area type for WPC.

[torotil]

Seems like in the new shape files the area_codes for Wales have changed. I’ve added

            if area_code.startswith('WP'):
                area_code = 'WA' + area_code[2:]

to mapit_gb/management/commands/mapit_UK_import_boundary_line.py:71 and that fixed it for me.

[dracos]

We imported 2022-05 Boundary-Line without receiving this error, so I am confused! And hoping I haven't broken something :-/ Thanks for letting us know. Ah, I guess as we had existing data with the same ONS GSS code, it matched on that and so did not have to try to create a new area.

[dracos]

So if they have renamed WAC and WAE to WPC and WPE (obliquely referenced in the release notes as a Welsh Parliament name change, but not in a way that meant I realised), question is whether we should maintain them as WAC/WAE (as your change would do), or add them as new Types. Seems silly to break WAC/WAE for no reason (it's not even as if any boundaries have changed), so your idea sounds good to me unless I've missed something. (We'd already renamed the descriptions of WAC/WAE to use Senedd in our MapIt some time ago.)

[dracos]

Though should we have done the same with EUR->ER for the English regions, hmm. Guess too late there.

Also if you don't know, North Ayrshire and Arran WMC is missing from the new Boundary-Line.

[torotil]

@dracos Thanks for confirming that that’s indeed a good fix … and also for the info about the WMCs. I guess you just had to raise the generation to get them from an earlier import, right?

Seems I also need to wrap my brain around adding the new councils (Cumbria, Somerset, …). As the new areas seem to be supersets of the existing ones this should be easily scriptable.

[dracos]

Yes, that's how I noticed, when there was one WMC left in the old generation :) Yeah, if you need them sooner, that's how you'd have to do it, I think (might eventually get around to doing #7 someday!). They'll then be in the May 2023 Boundary-Line, it sounds like.

https://github.com/mysociety/mapit/pull/386 - thanks for the suggestion, I decided probably better to be explicit about what's being replaced, just in case.