search

agrc / porter

UGRC tracks the additions, replacements, and deletions of SGID items (in the broadest sense of add, replace, or delete) through issues in this repository.
https://gis.utah.gov/documentation/policy/
MIT License
2 stars 0 forks source link

Remove and shelve outdated Utah School Board District data from Open SGID (remain in AGOL as shelved) #219

Closed gregbunce closed 1 year ago

gregbunce commented 1 year ago

Summary

A short summary of the situation.

This school board data/district is no longer active. As of Jan 2023, we are now using the latest districts and this one need to be removed from Open Data but made available in AGOL under the 'shelved' tag. Also, we should rename the item names in AGOL to be more descriptive and to better match our current naming convention for this layer. The remaining should be as follows:

Also, see this somewhat related porter issue in which we are deprecating the School Board District named 2012.

Migration Guide

This dataset has been replaced by Utah School Board Districts 2022 to 2032.

At this point, we should be migrating applications and users away from this outdated layer and onto the latest data.

Let's do a soft delete before we do a hard delete (aka: remove from Open Data) because of the many systems and users that were/are potentially using this data.

Action items

  1. Assign a person who should complete the task by replacing name with their github @name.
  2. Check [x] the box when the task is completed and add the date of completion.
  3. ~Strike~ out all items that do not apply.

Soft Delete

The purpose of the soft delete is to ensure that all of our users and applications have gracefully migrated off of the dataset. Soft deletes will remain in effect for 14 days. During this time, we will have the ability to restore the dataset to its original SGID offering(s). After these 14 days, the item is then ready for a hard delete.

Note: If this dataset is being replaced, then wait until the new data is publicly available before completing these steps:

Shelve/Static

Choose one based on situation.

Add record to table.

Hard Delete (to happen after 2/15/2023)

Hard deletes are final. It is recommended to complete the soft delete process before moving on to these steps. If you decide to skip the soft delete, note that you will need to incorporate some of those steps here.

:robot: Automation validation

  1. Assign yourself or someone to check the item by replacing name with their github @name.
  2. Check [x] the box and add the date of verification 2020/01/01 when the task is verified.
  3. ~Strike~ out all items that do not apply.

Are there service dependencies

Notification

Group Task Assignments

  1. Check [x] the box when you have assigned all the tasks relevant to your group.
gregbunce commented 1 year ago

/remind me to finish shelving on 2/10/2023

github-actions[bot] commented 1 year ago

@gregbunce set a reminder for 2/10/2023

gregbunce commented 1 year ago

@jacobdadams if you get a min, please look over my actions above (in the soft delete and shelve/static sections) to make sure I'm jiving with your processes. i don't want to blow anything up (though, it could be fun!).

mainly...

image

jacobdadams commented 1 year ago

Everything looks good, but the Auditor logs are telling me I missed a step with the soft delete- it doesn't like having the item id in both tables. @gregbunce, are you ok if I remove the itemid from the main meta table?

gregbunce commented 1 year ago

thanks for making sure all the wires are connected under the hood, @jacobdadams. sure, you can remove the itemid from the meta table. thanks. this layer was my dry run (aka template) on how to proceed with the other political district layers so, should I remove the itemid when moving ahead with the other layers (us cong; state senate; state house)? The workflow is pretty much identical to this one.

jacobdadams commented 1 year ago

Great, I've got the itemid removed from the meta table and I'll do a manual run of auditor once I get the conda environment set up.

And yeah, I think this works great moving forward. Let's just make sure we remove the itemid from the meta table once we put it in the shelved table.

gregbunce commented 1 year ago

Great, that works. also, i added you (@jacobdadams) to the task of removing the row from the meta table. I'm not sure the timing you'd like on that, so i figured i'll assign that one to you. feel free to remove it whenever you see it necessary (my workflow does not require that row there).

jacobdadams commented 1 year ago

We may want to double check with @steveoh on that- I feel like that table gets used for the clouddb stuff that loads items to the Open SGID.

steveoh commented 1 year ago

I believe as soon as you remove that row from the meta table that the table will be removed from the OpenSGID. I don't see any issues with this as from the description that appears to be what we want to do. Since we are putting this on the shelf, we only need to make sure the agol item isn't removed and everything else should be removed.

github-actions[bot] commented 1 year ago

:wave: @gregbunce, finish shelving

gregbunce commented 1 year ago

bumping the removal date in Internal db to 2/15/2022

agrc-conductor commented 1 year ago

conductor results for tasks - 219

check status
@gregbunce has completed 10 out of 14 tasks :no_entry:
@steveoh has completed 0 out of 3 tasks :no_entry:
@jacobdadams has completed 0 out of 1 tasks :no_entry:
@rkelson has completed 0 out of 1 tasks :no_entry:
agrc-conductor commented 1 year ago

conductor results for tasks - 219

check status
gregbunce has completed 14 out of 14 tasks :+1:
@steveoh has completed 0 out of 6 tasks :no_entry:
@jacobdadams has completed 0 out of 1 tasks :no_entry:
@nathankota has completed 0 out of 1 tasks :no_entry:
@rkelson has completed 0 out of 1 tasks :no_entry:
agrc-conductor commented 1 year ago

conductor results for tasks - 219

check status
gregbunce has completed 14 out of 14 tasks :+1:
@steveoh has completed 0 out of 6 tasks :no_entry:
@jacobdadams has completed 0 out of 1 tasks :no_entry:
@nathankota has completed 0 out of 1 tasks :no_entry:
@rkelson has completed 0 out of 1 tasks :no_entry:
steveoh commented 1 year ago

My validation tasks look good.

agrc-conductor commented 1 year ago

conductor results for tasks - 219

check status
gregbunce has completed 14 out of 14 tasks :+1:
steveoh has completed 6 out of 6 tasks :+1:
jacobdadams has completed 1 out of 1 tasks :+1:
@nathankota has completed 0 out of 1 tasks :no_entry:
@rkelson has completed 0 out of 1 tasks :no_entry:
agrc-conductor commented 1 year ago

conductor results for POLITICAL.UtahSchoolBoardDistricts2015

check status
internal sgid :+1:
meta table :+1:
stewardship :no_entry: There are multiple items with this name on rows 517, 517. Please remove the duplicates.