Continue shrinking the Internal SGID

gregbunce commented 6 months ago

Benefit

The benefit of continuing to shrink the Internal SGID is that we are not maintaining irrelevant data, and therefore this frees up our time to work on more important projects.

In the world of UGRC, this is an ongoing project that we should make progress on each quarter. That's the nature of of our business (keeping the data relevant).... datasets change, the stewards find better ways to serve them, etc.. In turn, we need to stay current and clean up the Internal so we're not serving outdated or irrelevant data.

For this quarter I propose the data team take a look at datasets that have not had an update since 2020. We flag those datasets and then establish a path forward for them. Some may need to be deprecated, others may become a part of the SGID Index b/c the stewards are now serving their own web service (data).

Acceptance Criteria

[ ] Document (begin documenting) what the SGID will look like over the next 3 to 5 years.
~- [ ] Create a list of datasets in the Internal SGID that have not had an update since 2020~ ~- [ ] Determine a path forward for those datasets (keep, deprecate, data-come-full-circle and steward has a web service, etc.)~

Notes

Other things to think about:

Regarding the website, what constitutes a data page versus simply an SGID index entry?

Risks

None

Issue Reference

5

gregbunce commented 5 months ago

Rick and I are meeting June 4th to discuss this and come up with a potential mini project for this in sprint 6

gregbunce commented 4 months ago

Rick and I decided to use some time in sprint 6 to create an outline on what the sgid looks like moving forward. We'll be working on that and then seeking feedback.

rkelson commented 3 months ago

We should create a list of feature layers that reside in AGOL only with no Internal layer. This idea is in response to removing State Fuel Sites from Internal while having it persist in AGOL and needing a strategy to backup AGOL only layers.

rkelson commented 3 months ago

ran this query on the SGID Index tab and it only came up with one layer, the Statewide parcel layer used in applications: storageType IN ('AGOL', 'cloud storage') And hostedBy = 'UGRC' And arcGisOnline = 'TRUE' And itemId IS NOT NULL

refs #325

steveoh commented 3 months ago

Statewide parcels are a bit unique since we can recreate them from data in the internal database.

gregbunce commented 1 week ago

rick and i had some good discussions this quarter. as a data team, we've all also discussed this during our 'sgid: a plan for the future" meetings.

gregbunce commented 6 days ago

informed from the rating system project

agrc / planning-queue