Fetching AOI name from the master list

[jdduh]

This issue is of the lowest priority.

This is related to one discussion in #11, which says "Change document titles. Replace stationName from pourpoint layer with nwccname from master_aoi_list.csv stored on AGOL. Use the station triplet from the pourpoint layer to look up the record in the .csv file (same link as retrieving annual runoff value). This is a different .csv from the runoff .csv. Replace spaces with underscores before using. Map and chart titles need to use this value too."

During a recent test on an atypical AOI, the BATCH tool retrieves the stationName of the "closest" point from the master list. We should set the search radius (for example 500 meters) so that only "matched" (instead of the closest) points are used to set the AOI name and report title. For AOI that doesn't have a match, we can put "N/A" in several data fields on the summary page.

Here is a list of these N/A data fields.

0. Filename: 00000001_XX_XXXX_Watershed-Report.pdf. The 01 is a counter to prevent output with the same file name.
1. Map title: Watershed Characteristics Report AOI not associated with a station
2. Streamgage station: N/A
3. Streamgage station name: N/A
4. Dealineated drainage area: unknown square miles (question: did we get the area from the pourpoint featureclass? If yes, then set the value to unknown.)
5. Annual runoff ratio: unknown

[jdduh]

The new version has resolved the "no change-match" issue and shows no issues with the previous results. There is one outstanding issue that's related to the master forecast station list but we don't have to deal with it right now (if we use the tool carefully). Here is a description of the issue: There are 8 locations with two coincident "forecast stations." 7 were in CA, which is not monitored by NWCC. The one (i.e., pair) in UT is for stations 09234400:UT:USGS and 09234300:UT:USGS (see the log below). NWCC wants to keep 09234400:UT:USGS, but the tool keeps changing it back to 09234300:UT:USGS when the attribute-match method is disabled. I have (had) to manually update the attribute.

3/17/2023 10:38:45 \New_AOIs_2022\09234400_UT_USGS_04132022 09234400:UT:USGS 09234300:UT:USGS Updated-Near (Is 09234400:UT:USGS)

A possible way to fix this is to check some attributes in the master FCSH list so that we can ignore some of the active forecast stations (and treat them as non-forecast stations). I'm going to close this case for now. Thanks for the quick response.

[jdduh]

For debugging purposes on AOIs with missing stationtriplet and stationname data (related to #40): I posted a "brand new" AK AOI that was created straight from BAGIS V3. The AOI is 15493000_AK_USGS_Debug.zip and can be found at the basins server's BAGIS aois folder. The file size is close to 60MB. The pourpoint to the actual forecast station location is around 2KM due to the flat terrain in the AK valley floors. As such, the batch tool/update station data tool would label this AOI as "not a forecast". The AOI's aoi_v and pourpoint data don't have the required the stationtriplet and stationname fields.

[lbross]

Since this is not exclusively an Alaska issue, I opened a new issue to handle basins for which there is no match on the FCST master list. There is also some complexity that we'll need to work through to clarify the requirements. Can you close this issue and we will work this problem on issue #43?

[jdduh]

This is the comment on the Update Forecast Station Data tool, more specifically, the merge all aoi_v polygons feature. As I mentioned earlier, the tool went through checking the stationTriplet data relatively quickly, but took a long time to merge the aoi_v. The problem seems to be related to the field_mapping parameter set for the Append tool (https://pro.arcgis.com/en/pro-app/latest/tool-reference/data-management/append.htm). The screenshots below show the time it took for Append complete the merge for each AOI.

Here is what it looks like at the beginning.

Here is what it looks like later.

Here is an example of the append parameters from the geoprocessing history.

Input Datasets K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Arkansas_chalk_ck_nr_nathrop_12062011\aoi.gdb\aoi_v Target Dataset K:\NWCC\2023_NWCC_ActiveForecast_AOIs\maps_publish\merge.gdb\merged_aoi_polygons Field Matching Type NO_TEST Field Map awdb_id "awdb_id" true true false 30 Text 0 0,First,#,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Arkansas_chalk_ck_nr_nathrop_12062011\aoi.gdb\aoi_v,0,30,30;stationTriplet "stationTriplet" true true false 255 Text 0 0,First,#,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Arkansas_chalk_ck_nr_nathrop_12062011\aoi.gdb\aoi_v,stationTriplet,0,255;stationName "stationName" true true false 255 Text 0 0,First,#,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Arkansas_chalk_ck_nr_nathrop_12062011\aoi.gdb\aoi_v,stationName,0,255 Subtype
Updated Target Dataset K:\NWCC\2023_NWCC_ActiveForecast_AOIs\maps_publish\merge.gdb\merged_aoi_polygons

The field_map value seemed to be accumulated in the subsequent runs, which could take ArcGIS Pro a long time to just parse the parameters. I checked the history of a latter Append process. Its field map parameter contains hundreds (if not thousands) of lines. See below for an example. I wonder if you can reset this parameter so that the Append tool won't take incremental time to complete.

Field Map awdb_id "awdb_id" true true false 30 Text 0 0,First,#,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Arkansas_chalk_ck_nr_nathrop_12062011\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Arkansas_eagle_nest_reservoir_inflow_12062011\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Arkansas_grape_ck_nr_westcliffe_12062011\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Arkansas_pueblo_reservoir_inflow_12062011\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Arkansas_rayado_ck_nr_cimarron_12062011\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Beaverhead_beaverhead_r_at_barretts_10302010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Beaverhead_big_hole_r_at_wisdom_10302010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Beaverhead_clark_canyon_inflow_10302010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Beaverhead_Ennis_Reservoir_Inflow_01132011\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Beaverhead_Hebgen_Reservoir_Inflow_01132011\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Beaverhead_Hyalite_Reservoir_Inflow_01132011\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Beaverhead_lima_reservoir_inflow_10302010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Beaverhead_missouri_r_at_toston_10302010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Bighorn_bighorn_r_at_kane_11032010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Bighorn_bighorn_r_nr_st._xavier_11022010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Bighorn_dinwoody_ck_nr_burris_11022010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Bighorn_greybull_r_nr_meeteetse_11032010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Bighorn_little_wind_r_nr_riverton_11022010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Bighorn_nf_shoshone_r_at_wapiti_11032010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Bighorn_shell_ck_nr_shell_11032010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Bighorn_wind_r_at_riverton_11022010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Klamath_sprague_r_nr_chiloquin_12062011\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Klamath_upper_klamath_lake_inflow_12062011\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Alaska_AOIs_2023\East_Kenai_Peninsula_Basin\15258000_AK_USGS_03272023\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Alaska_AOIs_2023\East_Kenai_Peninsula_Basin\15276000_AK_USGS_03272023\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Alaska_AOIs_2023\Kuparuk_and_Sagavanirktok_R_Basin\15896000_AK_USGS_03272023\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Alaska_AOIs_2023\Kuparuk_and_Sagavanirktok_R_Basin\15908000_AK_USGS_03272023\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Alaska_AOIs_2023\Tanana_R_Basin\15200280_AK_USGS_03262023\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Alaska_AOIs_2023\Tanana_R_Basin\15484000_AK_USGS_03272023\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Alaska_AOIs_2023\Tanana_R_Basin\15511000_AK_USGS_03272023\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Alaska_AOIs_2023\Tanana_R_Basin\15515500_AK_USGS_03262023\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Alaska_AOIs_2023\Yukon_R_Basin\15388960_AK_USGS_03272023\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Alaska_AOIs_2023\Yukon_R_Basin\15453500_AK_USGS_03282023\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\ColoradoCisco\blue_r_bl_green_mountainreservoir(2)_09072010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\ColoradoCisco\cochetopa_ck_bl_rock_ck_nr_parlin_09072010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\ColoradoCisco\colorado_r_nrdotsero(2)_09072010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\ColoradoCisco\dolores_r_at_dolores_09072010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\ColoradoCisco\gunnison_r_neargunnison(2)_09072010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\ColoradoCisco\gunnison_r_nr_grandjunction(2)_09072010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\ColoradoCisco\mcphee_reservoirinflow(2)_09072010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\ColoradoCisco\mill_creek_at_sheley_tunnel_nr_moab_09072010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\ColoradoCisco\paonia_reservoirinflow(2)_09072010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\ColoradoCisco\ridgway_reservoirinflow(2)_09072010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\ColoradoCisco\san_miguel_r_nr_placerville_09072010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\ColoradoCisco\slate_r_nr_crested_butte_09072010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\ColoradoCisco\tomichi_ck_at_gunnison_09072010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\ColoradoCisco\tomichi_ck_at_sargents_09072010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\ColoradoCisco\willow_ck_reservoirinflow(2)_09072010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Duchesne\ashley_ck_nr_vernal_09022010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Duchesne\duchesne_r_ab_knightdiversion(2)_09032010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Duchesne\duchesne_r_atmyton(2)_09032010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Duchesne\rock_ck_nr_mountainhome(2)_09032010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Duchesne\strawberry_r_nrduchesne(2)_09032010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Duchesne\upper_stillwater_reservoirinflow(2)_09032010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Duchesne\whiterocks_r_nr_whiterocks_09032010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\FlamingGorge\blacks_fork_nr_robertson_09072010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\FlamingGorge\fontenelle_reservoirinflow(2)_09072010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\FlamingGorge\hams_fk_blw_pole_ck_nr_frontier_09072010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\FlamingGorge\new_fork_r_nr_big_piney_09072010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Gila\gila_r_at_gila_09082010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Gila\gila_r_nr_solomon_09082010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Gila\san_carlos_reservoirinflow(2)_09082010\aoi.gdb\aoi_v,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\Gila\san_francisco_r_at_clifton_09082010\aoi.gdb\aoi_v,0,30,30,0,30,0,0,30,K:\NWCC\2023_NWCC_ActiveForecast_AOIs\GreatBasin\Carson\cars .....

[lbross]

Thank you for troubleshooting this issue. I was not cleaning out the stringBuilders that generate the filter text for each polygon merge. This is fixed for the next version of BAGIS-Pro 3.x.

[jdduh]

It seems that all the AOIs that were processed with the Batch tool are no long "ready" due to missing huc2 value in the AOI's pourpoint featureclass. Can you modify the "Forecast Station Data" tool so that when users use the "Update station data based on stationTriplet value" option, the tool also update the huc2 attribute? The huc2 value is now available in the stations_FCST_ACTIVE feature layer (and also the first two digits of the stations' awdb huc attribute).

[lbross]

The 'Forecast Station Data' tool DOES update the huc2 value with the "Update station data" option. If you aren't using the addIn that I posted yesterday afternoon, it is probably pointed at the USGS service which doesn't have the huc2. The stations_FCST_ACTIVE service should always supply the huc2 because it is populated as the first two digits of the stations' awdb huc attribute for new stations.

Once the updated addIn is using the stations_FCST_ACTIVE service, this problem should be resolved except for manual AOIs.