HHD Setup - Offline Capability Updates

[KCI-Ablowers]

Background

Due to limitations on data/hotspot usage at MDOT/SHA/whomever, HHD will need the BMP inspection workflow to be offline capable.

To-Do

Update or recreate the following features to be offline capable -

• [x] SWMFAC Inspection Tracker
• [x] NPDES Structures
• [x] NPDES Conveyance
• [x] NPDES SWMFAC
• [x] NPDES Access Point
• [x] NPDES Access Path
• [x] NPDES Drainage Areas

For the Structures, Conveyance, and SWMFAC - It seems it may be easier/safer to publish a new feature service that is offline-capable rather than update the public features. @johnshiu, I spoke with @talllguy about the public layers and he said there may be some issues with republishing the public NPDES layers with sync capabilities. Would the same apply to the SWMFAC Inspection Tracker feature service?

@talllguy The "Imagery Hybrid" basemap is already offline friendly, correct?

[talllguy]

The imagery hybrid is offline capable, yes. That is why it is the default--to support offline.

Making the public layers offline capable requires some technical configuration changes, which I recall is challenging for the scripted updates, do I have that right @johnshiu? We could make the feature service sync capable by enabiling archiving, if I recall correctly, or making it versioned. Another option is an AGOL sync. Since we've first encountered this problem, we've created the so-called AGOL syncs which can be made to go offline a little easier than working on the DMZ box (I recall there being some versioning challenges.) Perhaps we set up a once-in-a-while NPDES export to AGOL for use in the field.

Another option is to switch the field data to a tiled map service, though with performance implications I am not sure if that is a direction we should pursue.

Bottom line: 4 Options

1. Make existing services sync friendly
2. Make new DMZ hosted sync friendly service
3. Make new AGOL hosted sync friendly service mirror
4. Make new map service for tiling

I'd go for 2 or 3 if 1 is not viable.

[johnshiu]

I don't think there's technically any limitation for using the existing FLAT feature services offline with no editing capabilities. However, I tried and failed to download a map area using a test map I created. Can someone else also confirm that the existing available services don't work offline? The complexities Elliott mentions with the offline capability are usually only for editing, but since we are not allowing for inventory edits, I think these requirements are relaxed.

If the existing read-only views don't work, then I would suggest option 3 from the list. I don't think we can rely on the DMZ service performance as much as we can with AGOL. I also don't think AGOL inventory will be updated that frequently to require updates, so even if the sync script is not perfect (I haven't done a source to AGOL sync of this size with the NPDES dataset), we could just manually reupload the changes as a last resort.

[KCI-Ablowers]

I tried a few different ways with the public features and couldn't get the "offline area" option to show up in Collector either. I'm fine with any of the options above as long as you guys are good with it and it works. I'm not sure the exact field days, but I think we may be going out at the end of this week or the beginning of next.

[johnshiu]

Got it; thanks for confirming, Andrew. Let me setup a feature service endpoint in AGOL that mirrors the FLAT feature classes and see if you have any trouble with the offline functionality.

The only complication I can think of is that the symbology would probably have to be recreated somehow in AGOL, and I'm not sure of an easy way of doing that. Maybe something like opening the original MXD in Pro and then changing the datasource of the layer to the feature service in AGOL, and then publishing the layer in AGOL through Pro? I haven't done any of that before, so it's purely conjecture.

[talllguy]

John, thanks for testing and clarification on the technical issues at play. I am glad we'll go with the AGOL sync.

Regarding symbology, I have a AGOL-compatible stylex file somewhere but it would not be terrible difficult to redo the symbology directly in AGOL in the visualization settings. That UI allows us to update the symbol with a graphic. That would be easy if the periodic updates wouldn't wipe the symbology. If the symbology does reset each sync, then I'd agree that it is worth redoing as a publish via Pro. Perhaps there's even a way to do that with some of the publishing capabilities in pro via ArcPy.

[talllguy]

Update on symbology, I added the Pro-compatible NPDES Symbology.stylx file as well as source layers and images in the symbology folder on this repo for safekeeping and experimentation.

[johnshiu]

@talllguy @KCI-Ablowers, I've added the feature service with the flattened layers here: (craig.mackowiak8): https://maryland.maps.arcgis.com/home/item.html?id=c1beb33517e249779b8d2e9dc4bf1f29

Can you please try it out and see if it will go offline for you? If it does, then I think we can go ahead and add the symbology, etc. I'll try to figure out a way to do the updates reliably, but again, I'm not sure if updating 300k+ features is going to work well via the AGOL service endpoint. Another option may be to just "Update" the file geodatabase manually with a refreshed dataset periodically, but I haven't tried doing that before on a live feature layer before.

[KCI-Ablowers]

@johnshiu I jumped on as the SHA DGD account and turned on the "Sync" option for the service above. I can now create an offline area in Collector. I had to add an editable feature (Access points in this case) for the map to show up.

I think the periodic update would be fine at this point. I assume all of the FY21 BMPs should be in NPDES prod and in this flat feature service. Any inventory or attribute updates will occur outside of Collector/AGOL so those will probably be stacked up for a rainy day.

@talllguy The drainage area layer in the WTB map (https://utility.arcgis.com/usrsvcs/servers/590adefae5b84737ba80c744d4ffe46c/rest/services/NPDES/NPDES_Drainage_Area_Maryland_MDOTSHA/MapServer) does not appear to be offline capable. I was going to try the other layer in the KCI_1 map but it says my andrew.blowers account doesn't have permission to access.

[KCI-Ablowers]

Our first training day is set for Thursday, 10/1. As of right now, we will be visiting the same sites across all three of the planned training days. I am going to set up and test everything for "HHD_1" since only one person will submit the inspection records. The rest can get set up sometime next week.

[johnshiu]

@KCI-Ablowers and @talllguy, I've updated the symbology on the service so that it matches that of the originals. I found the best way to do this is to go to an existing, reference map service and retrieve the "drawingInfo" parameter from the JSON: https://maps.roads.maryland.gov/arcgis/rest/services/NPDES_Next_Gen/NNG_Inventory_V_KCI_Dev_20181228/FeatureServer/0?f=pjson

Then, use that "drawingInfo" and copy it to the UpdateDefinition admin endpoint on the AGOL feature service: https://services.arcgis.com/njFNhDsUCentVYJW/ArcGIS/rest/admin/services/NPDES_Default_Layers/FeatureServer/1/updateDefinition

You need to surround the JSON element in curly braces, so something like this: {"drawingInfo": { "renderer": { "type": "uniqueValue", "field1": "CONV_TYPE", "field2": null, "field3": null, "defaultSymbol": { "type": "esriSLS", "style": "esriSLSSolid", "color": [ 205, 46, 49, 255 ], "width": 1.5 }, "defaultLabel": "Unknown", "uniqueValueInfos": [ { "symbol": { "type": "esriSLS", "style": "esriSLSDash", "color": [ 56, 168, 0, 255 ], "width": 3 }, "value": "DITCH", "label": "Ditch", "description": "" }, { "symbol": { "type": "esriSLS", "style": "esriSLSDash", "color": [ 255, 0, 197, 255 ], "width": 1 }, "value": "CONNECTOR", "label": "Hydraulic Connector", "description": "" }, { "symbol": { "type": "esriSLS", "style": "esriSLSSolid", "color": [ 0, 0, 0, 255 ], "width": 2 }, "value": "PIPE", "label": "Pipe", "description": "" } ], "fieldDelimiter": "," }, "transparency": 0, "labelingInfo": null }}

Then, use the Refresh admin option on the layer to ensure the changes go through.

The new feature layer containing all the NPDES data can be found here: https://maryland.maps.arcgis.com/home/item.html?id=e0c47582add04e40b61f9b16005bbde2

[KCI-Ablowers]

Thanks, @johnshiu. I was able to successfully download an offline area with the new default features. However, I ran into an issue with the offline area if I had the "SWFMAC Inspection Tracker" FS in the web map. I thought it worked previously, but I never tried to actually download the area (just looked for the option). For now, that is not included in the HHD_1 web map.

[johnshiu]

@KCI-Ablowers, I think it's because the tracker feature service is hosted at SHA: https://maps.roads.maryland.gov/arcgis/rest/services/NPDES_Next_Gen/SWMFAC_Insp_Tracker_Field_MDOTSHA/FeatureServer

Since it needs to sync with the NPDES source tables, I'm not sure if there's an easy way to incorporate this. We could push it to AGOL, I suppose, but then we would need to sync it back to SHA as well as reconcile with the NDPES source tables. It sounds a bit complicated.

Longer term, maybe the ArcGIS Portal release could relieve some of these complexities with the AGOL syncing.

[KCI-Ablowers]

@johnshiu, I think we leave it as is for now. My thought is to set up an "Edit App" for the access and tracker updates. The offline workflow is clunky at best so the fewer layers involved the better. We will have to reiterate to Kiona and Co. that the tracker should be updated as often as possible.

-- Edit Or, we ask them to make the updates when they are connected to wi-fi on their device of choice. @talllguy we also may need to rethink the basemap. The downloaded areas are sizeable and take a while on my wi-fi with the Imagery Hybrid.

[KCI-Ablowers]

SWMFAC Inspection tracker will be in the "Online" version of the web map. Closing.