Replace drive links with open data

[steveoh]

We are taking advantage of the backend provided by open data to provide current downloads in useful formats without having to manage duplicate files. Checkout the readme for the syntax of hub downloads.

What page is the problem visible

911

• [x] 911 (not in open data, but moved download to agrc_public_share)

address-geocoders-locators

• [x] address-geocoders-locators (not in open data, but moved download to agrc_public_share)

boundaries

• [x] citycountystate
• [x] zip-codes
• [x] wilderness
• [x] school-districts
• [x] conservation-districts

cadastre

• [x] plss

demographic

• [x] census

economy

• [x] taxingareas
• [x] enterprise-zones

energy

• [x] uranium
• [x] renewable-energy
• [x] energy-generation
• [x] oil-gas
• [x] coal

environment

• [x] drinking-water-systems-protection-zones
• [x] deq-land-related-contaminant-cleanup-sites
• [x] deq-air-related-contaminant-cleanup-sites

farming

• [x] grazing-improvements

geoscience

• [x] quaternary-faults
• [x] geochronology
• [x] soil
• [x] alluvial-fans
• [x] avalanche
• [x] landslides
• [x] earthquakes
• [x] aquifers

health

• [x] health-districts
• [x] emergency-regions
• [x] health-small-statistical-areas
• [x] health-care-facilities

indices

• [x] auto-correlated-dem-indices
• [x] contours-indices
• [x] usgs-topo-indices
• [x] national-grid-index
• [x] usgs-dem-indices
• [x] lidar-indices
• [x] aerial-photography-indices

indices are hidden and not in open data. the plan is to move them to the agrc share folder

location

• [x] u-s-postal-service
• [x] gnis
• [x] building-footprint
• [x] address-data

planning

• [x] water-related-land

political

• [x] 2002-2011-house-senate-congressional-districts
• [x] 2012-2021-house-senate-congressional-districts
• [x] judicial-districts
• [x] voter-precincts

recreation

• [x] golf-courses
• [x] trails
• [x] local-parks
• [x] boat-ramps
• [x] ski-areas

society

• [x] state-facilities
• [x] places-of-worship
• [x] cemeteries
• [x] blm-field-offices-forest-service-stations
• [x] liquor-stores
• [x] schools-libraries
• [x] state-fuel-sites

transportation

• [x] air
• [x] transit
• [x] street-network-analysis (not in open data, but moved download to agrc_public_share)
• [x] roads-system
• [x] railroads

utilities

• [x] broadband-internet
• [x] telephone-infrastructure-and-rural-telcom-boundaries

water

• [x] flood-areas
• [x] historic-lake-bonneville
• [x] watersheds
• [x] lakes-rivers-dams
• [x] water-rights-regions

Expected result

All drive.google.com should be replaced or if not available the data should be moved to the public share re: #1297

[gregbunce]

thanks for getting this going. i will move 911 (it's an fgdb), the roads (shp that's too large for open data), the address geocoders, and the network datasets to the public share.

also, I'll remove the remaining transportation and trails data links.

I've added it to my list of items for the next sgid road update (first week of Nov)

matt and I are meeting with esri next week to chat about interoperability etc, and I plan on bringing up the file size download limit (that is affecting the roads.shp)

[rkelson]

I plan to tackle Society next

[brigsz]

Okay, I looked things over and I see the pattern of how this is done. I need to speak with someone to make sure I am not going to make a mess. It seems pretty straightforward, putting in the hub: etc

[rkelson]

what is tricky is some pages can have a combination of Hub: download links and related resource that are HTML links to the open data pages and possibly agrc_public_share links for things not in Open Data.

[rkelson]

I am going to work on Boundaries next

[rkelson]

we could probably close this. Geocron is Zach's problem and fuel sites is a WIP

[steveoh]

Ok, I'd rather wait until it everything is complete to close this. I'll add the waiting tag for @ZachBeck and the state fuel site project.

[ZachBeck]

I haven't been involved with the fuel site project.

[steveoh]

I haven't been involved with the fuel site project.

Yes I am aware. But rick called you out on geochronology which is why I mentioned you.

[ZachBeck]

I thought I did that here https://github.com/agrc/gis.utah.gov/pull/1488

[steveoh]

Ok, I think I need to be brought back to the same page. I was under the impression that Gordon was going to host all of those datasets on their arcgis online which would then make all of those urls incorrect. Do we need to have a quick chat to move forward with this?

[ZachBeck]

Shoot, you're right. We were going to start with quaternary faults but I'll have him share his geochronlogy layers with our Geoscience group and then I'll delete our geochronology from agol if his looks good. Last time I checked no one has been using our geochronology so I don't feel bad about deleting it.

[steveoh]

Yah since the web pages haven't been published and we didn't introduce the services through porter, i am also ok with that decision

[stdavis]

These links are going to have to be updated to use the agol id rather than the item title at some point. Seems like they are redirected at the moment but I'm not sure how long those will last.

Here's a search that shows them: https://github.com/agrc/gis.utah.gov/search?q=https%3A%2F%2Fopendata.gis.utah.gov%2Fdatasets%2Futah

[steveoh]

interesting choice esri. most of these can be handled by the include and front matter but these static ones will need to be done manually. Have you spoken to anyone about the redirect longevity?

[stdavis]

Have you spoken to anyone about the redirect longevity?

No. I noticed that it was redirecting to the id-based routes today. I did notice that this one is already broken: https://opendata.gis.utah.gov/datasets/utah-oil-and-gas-well-locations/explore, although I'm not 100% sure that was the original URL. That AGOL item is a bit of a mess right now that I'm trying to get straightened out with OGM.

[steveoh]

item id is used as a fallback in case esri can’t generate the human readable url. There are organization configurations that prevent esri from ever generating the human readable url (e.g. anonymous access is disabled for the org) and there are times when our jobs system is catching up and hasn’t generated the url yet.

esri noticed many instances recently displaying ids when they were expecting human readable urls, and they eventually caught up

relatedly esri is working on a couple of fixes for redirects not working as expected, particularly for the old table routes and between /about and /explore

Based on this, I think you might have been noticing slow jobs and we should be ok?

[stdavis]

Interesting. Maybe we are ok. I noticed that all of the links in search results (https://opendata.gis.utah.gov/search) are id-based.

[stdavis]

@ZachBeck & @steveoh: So is Geochronology finished? Can this issue be closed?

@steveoh: What else needs to happen with the state fuel sites that you mentioned?

[ZachBeck]

Geochronology is good to go

[steveoh]

I have zero memory of state fuel sites. @rkelson whats the WIP status with that?

[rkelson]

I don't recall any WIP

On Mon, Jul 12, 2021 at 9:39 AM steveoh @.***> wrote:

I have zero memory of state fuel sites. @rkelson https://github.com/rkelson whats the WIP status with that?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/agrc/gis.utah.gov/issues/1521#issuecomment-878383805, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPJ43MTLCUQW7ZCXZJL3PTTXMEEHANCNFSM4SPMVLCA .

[stdavis]

I think that this issue can be closed then. Feel free to re-open if anyone feels differently.

[steveoh]