Updating LRC boundaries

[kitenetter]

ALERC would like to send us a new set of boundaries for LRCs across the UK (I think most of the changes apply to Scotland). What is the best format for them to send (shapefile?) and when are we likely to be able to import new boundaries? Are there any issues arising from such changes, e.g. records having to be reassigned to a different boundary?

(Such updates will mean that verifiers using the LRC boundaries will have access to a different set of records, but that is an inevitable result of boundary change.)

[johnvanbreda]

SHP is fine, ideally in OSGB 1936 projection. I think last time I had to split the file into lots of small ones as anything over a few MB is going to be tricky to upload. I presume these boundaries will replace the previuos ones? In which case we will indeed need to get the spatial indexing tool to reallocate all the records to the correct boundary in the new set. After the problems with the TWIC boundary we need to be aware that a) this can take a very long time, and b) any glitches in the boundary geometries need to be ironed out first. Also, David allocated the last import of SHP boundaries to Biren, I'm not sure if he'd want to do the same this time.

[kitenetter]

@johnvanbreda: Biren has done some work to clean the new set of LERC boundaries that we received from ALERC. The new set of boundaries is much more accurate than the old (see images below), but there is a new issue to address in that ALERC have now removed those counties that do not have an LERC, leaving gaps for places such as Essex and parts of Scotland.

What are the implications if we move to the new boundaries? One is that we may have some verifiers set up to use filters based on an LERC boundary that no longer exists (but we could probably reassign these to vice-counties). But more generally is there a problem with using a set of LERC boundaries that don't cover the whole country?

Options seem to be:

• Import the new boundaries and accept that there are gaps for some counties
• Add in our own polygons to fill the gaps
• Use a set of modern administrative counties (e.g. from OS data) instead of LERC boundaries
• Use a set of modern administrative counties (e.g. from OS data) as well as LERC boundaries (i.e. add a new set of boundaries so that people can choose one of the other)

Old ALERC boundaries (the ones we are currently using):

New ALERC boundaries:

[johnvanbreda]

Rather than the option of switching to modern county boundaries, we could use the vice county boundaries for areas where there are gaps? If the point of using LRC boundaries at all is to allow verifiers affiliated to an LRC to stick to their boundary as an alternative to the VC recorder network, then there doesn't seem much point in filling the gaps. If there is no LRC affiliation then we may as well stick to VC boundaries?

Looking at the images above, won't the lack of a buffer out to sea be a problem? In fact I know it would be e.g. for ERCCIS.

There are a few ways we could handle the update, but I think importing these boundaries as a new layer then re-indexing the whole lot could be quite an intensive task so we might want to avoid that, especially as verifiers would not be able to work in the meanwhile. An alternative would be to update the boundaries for existing location records, then re-index them one at a time. We'll need to write some scripts to do this of course. Any new LRC boundaries (if there are any - wishful thinking!) would simply be added to the locations list. Any deleted ones could be moved to a different locations layer so they aren't lost (e.g. an "expired LRC boundaries" layer) and their records would be removed from the spatial index. If there are any verifiers who use these expired LRC boundaries in their filters we'd need to review them on a case by case basis.

[kitenetter]

Our set of LERC boundaries does need updating to reflect recent changes, especially in Scotland, but I don't think we have fully resolved the questions raised by John above.

The most recent ALERC shapefile update is stored in: W:\PYWELL_SHARED\Pywell Projects\BRC_BRC_projects\iRecord\Boundaries\LERC Boundaries.zip

ALERC requested that we should completely replace the LERC boundaries currently in use.

Will update following meeting with ALERC on 9 August 2018.

[kitenetter]

We now have a new shapefile for the LERC boundaries: W:\PYWELL_SHARED\Pywell Projects\BRC_BRC_projects\iRecord\Boundaries\LERC british_isles_june_2018.zip

However, there remains the issue of the buffer out to sea. I'll discuss this with others at CEH to find a solution.

No further action required until this is resolved.

[kitenetter]

Awaiting response from ALERC in case anything has changed since we last looked at this.

[kitenetter]

Tom Hunt confirms that there have been no further changes in the ALERC boundaries, so we are fine to go with the set here: W:\PYWELL_SHARED\Pywell Projects\BRC_BRC_projects\iRecord\Boundaries\LERC british_isles_june_2018.zip

Actions required:

1. Ensure that the shapefiles contain a single (multi-part if needed) polygon for each LERC.
2. Edit the polygons for coastal LERCs to add a three-mile buffer out to sea, applying an arbitrary boundary between adjoining LERCs on the seaward side. (ALERC would be grateful for a copy of this layer once the extension has been added.)
3. Compare the new LERC layer with the one currently in iRecord to assess and resolve if there are any complications where LERCs may have been split or combined.
4. Import the new files into iRecord.

Are there are any further actions needed to re-index records against the new boundaries?

[burkmarr]

@kitenetter I've updated the LERC boundaries from a slightly cleaned version of what Tom sent through yesterday. Have a look and let me know if you have any concerns about me changing the live site.

Note that on dev I did this in a slightly awkward way which means LERC boundaries whose names changed were removed and replaced with new entities with the new names. But that means that if one of those locations whose name changed is referenced, e.g. by a filter, it will no longer be found. On the live site I will instead rename existing boundaries to match new names before updating so that they will keep their original IDs and anyone referencing them will be okay.

[burkmarr]

Note also that these locations from Tom file have a 12 nautical mile buffer.

[kitenetter]

@burkmarr I can't get dev iRecord to work (HTTP ERROR 500). From what you've said I would expect it to be okay to update on the live site, but was there anything in particular you wanted me to check?

Before doing the live update, could you let me know if there are any LERCs in the old layer that do not appear in the new one (and also if there are any in the new that weren't in the old) - thanks.

[BirenRathod]

@kitenetter No problem of accessing this http://test.brc.ac.uk/irecord_dev/ site.

[kitenetter]

Thanks @BirenRathod - at the second attempt it was fine, not sure why it got stuck first time.

@burkmarr I've looked at a few of the boundaries and all seems as expected, so unless there are any particular areas to check I think we can proceed. Will you update your previous map change notes on the forum when it's done?

[burkmarr]

@kitenetter the closer you look at this, the more difficult it becomes!

There are some very substantial changes to boundaries, particularly in Scotland. Types of change include:

• elimination of overlapping boundaries between neighbouring LERCs
• subsumption of some LERCs within a bigger neighbour
• new LERCs
• removal of former LERCs

The boundaries of most of the LERCs have changed and I will update all LERC boundaries to ensure that we are using the very latest (this is what 'update' means in the change column of the table below).

Where an existing LERC is going to be replaced by one with a different name that covers substantially the same area, I will first update the name of the existing location in the Warehouse before carrying out the boundary update. That will mean that, despite the change in name, the location keeps the same key. That is indicated by 'change name' in the table. The name that will be changed is the first LERC in the column that starts 'Old name...' (i.e. before any parentheses).

Some LERCs in the new layer can't be easily matched to LERCs in the existing layer. They may be entirely new (e.g. some of the Channel Islands) or they may overlap an existing LERC which is already assigned to another LERC in the new layer. So these LERCs can be considered new and will be added to the locations - generating new keys ('add' in the table).

LERCs in the existing layer which are no longer represented in the new layer, either because they have been remove or because they have been subsumed into a larger neighgour, will have to be deleted. These are probably the ones which we need to be most concerned with since any filters etc which reference them will no longer work.

@kitenetter - can you look at those ones in particular in the table just in case you spot any potential problems. @johnvanbreda - can you think of any cases where the deletion of a location that is referenced elsewhere in the system could cause more than a mere inconvenience to a recorder? In other words should be check anything out before deleting these? (I will do any deletion through the Warehouse GUI.)

Change type	Proposed location action	New name	Old name (other overlapping old LRCs)
new lrc	add	Alderney Records Centre
new lrc	add	Guernsey Biological Records Centre
new lrc and meaning change	update and change  name	Outer Hebrides Biological Recording	Not covered by an LRC
meaning change	update	Highland Biological Recording Group	Highland Biological Recording Group (Skye Environmental Centre)
name and meaning change	update and change name	South West Scotland Environmental Information Centre	Dumfries & Galloway Environmental Resources Centre (Arran BRC, Ayrshire BRC)
name and meaning change	update and change  name	The Wildlife Information Centre	The Wildlife Information Centre for the Lothians and the Borders (Central Scotland Wildlife Information Service)
name and meaning change	update and change  name	Glasgow Museums Biological Records Centre (core area)	Glasgow Museums Biological Records Centre (South Lanarkshire Council, North Lanarkshire Biological Records Centre)
name and meaning change	add	Glasgow Museums Biological Records Centre (non-core area)	(Glasgow Museums Biological Records Centre)
name and meaning change	add	Argyll Biological Records Centre	(Glasgow Museums Biological Records Centre)
name and meaning change	add	Scotland No LERC Present
no longer present	remove		Perth Museum Biological Records Centre
no longer present	remove		NatureBase (Dundee Museum)
subsumed into neighbour	remove		Skye Environmental Centre
subsumed into neighbour	remove		Falkirk Museum
subsumed into neighbour	remove		Arran Biological Records Centre
subsumed into neighbour	remove		Ayrshire Biological Records Centre
subsumed into neighbour	remove		Central Scotland Wildlife Information Service
subsumed into neighbour	remove		South Lanarkshire Council
subsumed into neighbour	remove		North Lanarkshire Biological Records Centre
name change	update and change  name	Orkney Wildlife Information and Records Centre	Orkney Biological Records Centre
name change	update and change  name	Shropshire Ecological Data Network	Shropshire Wildlife Trust
name change	update and change  name	Rotherham BRC	Rotherham Biological Records Centre
name change	update and change  name	Environmental Records Information Centre North East	North East Environmental Records Information Centre
name change	update and change  name	Norfolk Biodiversity Information Service	Norfolk Biological Records Centre
name change	update and change  name	Suffolk Biodiversity Information Service	Suffolk Biological Records Centre
name change	update and change  name	RECORD - Biodiversity Information Centre for Cheshire, Halton, Warrington & Wirral.	Biodiversity Information Centre for Cheshire, Halton, Warrington & Wirral
name change	update and change  name	Greater Manchester Local Record Centre	Greater Manchester Ecology Unit
name change	update and change  name	Surrey Biodiversity Information Centre	Surrey Biological Records Centre
name change	update and change  name	Herts Environmental Records Centre	Hertfordshire Biological Records Centre
name change	update and change  name	Islay Natural History Trust	Islay Natural History Trust Wildlife Information Centre
name change	update and change  name	Warwickshire Biologcal Record Centre	Warwickshire BRC
name change	update and change  name	ERC for Cornwall and the Isles of Scilly	Environmental Records Centre for Cornwall and the Isles of Scilly
name change	update and change  name	Lancashire Environmental Records Network	Lancashire Biological Records Centre
name change	update and change  name	Manx Biological Recording Partnership	Isle of Man Biological Records Centre
name change	update and change  name	Biodiversity Information Service for Powys & Brecon Beacons National Park	Biodiversity Information Service for Powys and Brecon Beacons National Park
none	update	Greenspace Information for Greater London	Greenspace Information for Greater London
none	update	Staffordshire Ecological Record	Staffordshire Ecological Record
none	update	West Yorkshire Ecology	West Yorkshire Ecology
none	update	Sheffield Biological Records Centre	Sheffield Biological Records Centre
none	update	Barnsley Biological Records Centre	Barnsley Biological Records Centre
none	update	Doncaster Biological Records Centre	Doncaster Biological Records Centre
none	update	Northamptonshire Biodiversity Records Centre	Northamptonshire Biodiversity Records Centre
none	update	EcoRecord	EcoRecord
none	update	Buckinghamshire and Milton Keynes ERC	Buckinghamshire and Milton Keynes ERC
none	update	Kent and Medway Biological Records Centre	Kent and Medway Biological Records Centre
none	update	Gloucestershire Centre for Environmental Records	Gloucestershire Centre for Environmental Records
none	update	Shetland Biological Records Centre	Shetland Biological Records Centre
none	update	Isle of Wight Local Records Centre	Isle of Wight Local Records Centre
none	update	Cofnod - North Wales Environmental Information Service	Cofnod - North Wales Environmental Information Service
none	update	Bristol Regional Environmental Records Centre	Bristol Regional Environmental Records Centre
none	update	Fife Nature Records Centre	Fife Nature Records Centre
none	update	South East Wales Biodiversity Records Centre	South East Wales Biodiversity Records Centre
none	update	Herefordshire Biological Records Centre	Herefordshire Biological Records Centre
none	update	Thames Valley Environmental Records Centre	Thames Valley Environmental Records Centre
none	update	Bedfordshire and Luton Biodiversity Recording and Monitoring Centre	Bedfordshire and Luton Biodiversity Recording and Monitoring Centre
none	update	Leicestershire and Rutland Environmental Records Centre	Leicestershire and Rutland Environmental Records Centre
none	update	Derbyshire Biological Records Centre	Derbyshire Biological Records Centre
none	update	Sussex Biodiversity Record Centre	Sussex Biodiversity Record Centre
none	update	Hampshire Biodiversity Information Centre	Hampshire Biodiversity Information Centre
none	update	Dorset Environmental Records Centre	Dorset Environmental Records Centre
none	update	Devon Biodiversity Records Centre	Devon Biodiversity Records Centre
none	update	Somerset Environmental Records Centre	Somerset Environmental Records Centre
none	update	Merseyside BioBank	Merseyside BioBank
none	update	West Wales Biodiversity Information Centre	West Wales Biodiversity Information Centre
none	update	Centre for Environmental Data and Recording	Centre for Environmental Data and Recording
none	update	Cumbria Biodiversity Data Centre	Cumbria Biodiversity Data Centre
none	update	North East Scotland Biological Records Centre	North East Scotland Biological Records Centre
none	update	North & East Yorkshire Ecological Data Centre	North & East Yorkshire Ecological Data Centre
none	update	Lincolnshire Environmental Records Centre	Lincolnshire Environmental Records Centre
none	update	Nottinghamshire Biological and Geological Records Centre	Nottinghamshire Biological and Geological Records Centre
none	update	Essex Wildlife Trust biological records centre	Essex Wildlife Trust biological records centre
none	update	Cambridgeshire and Peterborough Environmental Records Centre	Cambridgeshire and Peterborough Environmental Records Centre
none	update	Jersey Biodiversity Centre	Jersey Biodiversity Centre
none	update	Worcestershire Biological Records Centre	Worcestershire Biological Records Centre
none	update	Wiltshire & Swindon Biological Records Centre	Wiltshire & Swindon Biological Records Centre

[kitenetter]

@burkmarr could you let me have the location_ids for the old polygons in that table so that I can check against the verification filters - thanks.

[burkmarr]

@kitenetter - do you just want the ones identified for deletion?

[burkmarr]

Here they are:

id	name
1434	Perth Museum Biological Records Centre
1440	Naturebase (Dundee Museum)
1410	Skye Environmental Centre
1420	Arran Biological Records Centre
1425	Ayrshire Biological Records Centre
1444	Central Scotland Wildlife Information Service
1431	South Lanarkshire Council
1432	North Lanarkshire Biological Records Centre

[burkmarr]

I just notice this comment from @johnvanbreda from a couple of years ago in this issue:

update the boundaries for existing location records, then re-index them one at a time. We'll need to write some scripts to do this of course. Any new LRC boundaries (if there are any - wishful thinking!) would simply be added to the locations list. Any deleted ones could be moved to a different locations layer so they aren't lost (e.g. an "expired LRC boundaries" layer) and their records would be removed from the spatial index. If there are any verifiers who use these expired LRC boundaries in their filters we'd need to review them on a case by case basis.

@johnvanbreda - that advice about deleted boundaries sounds reasonable. Do you still recommend this as a way forward. Has the re-indexing comment been superseded?

[johnvanbreda]

@burkmarr this comment predates the work queue which should handle the re-indexing, though I can't guarantee that updates as complex as this won't affect server performance. The idea of moving locations into a new layer for expired LRC boundaries means that if any records point directly at these locations then the information isn't lost. It might be that there aren't any direct links but at least doing it this way you are safe.

[kitenetter]

@burkmarr none of the to-be-deleted boundaries are in use in our verification filters. As you say, it is possible that they have been included in individual users' saved filters, but I think as long as we explain in the forums what is happening that's fine - if anyone gets stuck they can contact us for support.

There are plenty of good reasons for doing this update of course as it will ensure that records end up with the correct LERC and are used for protection of local sites, so I think the benefits from proceeding outweigh any additional support we may have to provide.

[burkmarr]

How about having a general layer for "expired locations" rather than just expired LERCs? That way if a similar situation arises with other location types, we can just change their type to this rather than delete them. For example if we retire the Yorkshire and Sussex aggregate VC boundaries we could just change their types to this too. I imagine there's very little cost to hanging onto these polygons rather than deleting them.

@kitenetter @johnvanbreda do you agree with that as a way forward. Shall I go ahead and do it?

(My plan is to use the Warehouse gui to add a new type in the locations list and then editing the existing locations to change their type to this.)

[johnvanbreda]

@burkmarr I agree, that sounds fair. Is it worth adding the location type as a note in the comments or something, so we can disentangle the expired locations if we ever need to?

[burkmarr]

@johnvanbreda - will do.

[burkmarr]

I've completed this work. 'Retired' LRC boundaries moved to a new location type called 'Expired boundaries' with a comment to indicated that they were, prior to today, of the 'LRC Boundary' type. I also added a comment to all renamed LRC boundaries specifying what their name was prior to today. All boundary geometries were successfully updated and five new boundaries added.