Table with SNOTEL and SNOW COURSE sites

[lbross]

Separate page in the report behind the title page. Fields to include: station name, elevation, station triplet, begin date

[lbross]

We may want to consider including lat and long in this table. Although the symbols for the sites appear on the maps to show the location, there is nothing to tie each symbol to its site information. By including the lat/long, it could help the analyst determine which site is which by relative position. What do you think?

[jdduh]

Make sense. Please include lat and long in the site table.

[lbross]

I added this a second page behind the title page for the watershed and site analysis reports. I just posted a sample for Elwha to the NWCC portal. I noticed a problem where the roads layer is duplicated and am working on that. But you can review the sites table.

[jdduh]

NWCC has provided additional feedbacks on the Basin Analysis Report. I will create new issues for these updates. Here is the one related to the stations table. Add slope (% slope) and aspect (azimuth angle and direction description) to the station table. You can use the Extract Values to Point tool to get the data from the slope and aspect rasters. The aspect direction description is depended on the number of directions (4, 8, or 16).

[lbross]

Is it OK to use the slope and aspect zones layers with the extract values to points tool? I can then look up the zone number against the definition of the slope or aspect classes to get the common name. Examples: Flat, N, NE, E ...

Also, this tool is not the speediest. Is it OK if I run the tool after clipping the sites layer and calculating the aspect or slope zones? This way I would just have to look up the value when the report is generated. Would it be helpful to save this value on the snotel or snow course sites layers like we do the name and elevation? Otherwise I might store it in the analysis.xml.

[lbross]

Use 'Extract Multivalues to points' or 'Extract Values to Table' or instead. Allow multiple input rasters. Multivalues tool will add the values to the Feature Class. Save aspect raw number and description (Flat, N, NE, E ...). Save raw value of slope, not category. These values will be saved in the analysis.xml file.

[lbross]

Ugh!! I thought I had this working but it fails on CS3. I am getting an error with the Extract Values to Table gp tool. It won't run interactively either. When I add the third raster layer in the UI it gives me an error about that layer not having the same spatial reference. But I looked at all the layers and the spatial reference looks fine. I wonder if this is a bug with the new version of Pro? CS3 is running 2.8.1 and my laptop is running 2.6.3.

Can you see if you can run the tool both on your laptop and on CS3? Either of the sites layers is the feature layer and I am extracting from aspect, slope, and aspzone. Thanks!

[jdduh]

I got the same error message in Pro 2.8.1 with our AOI data. I then used the AOI dem to create slope and aspect layers. These outputs seem to work fine. I believe that the rasters in a BAGIS AOI don't have the same extent due to that we used different buffers to perform different analyses. After viewing the output table of the Extract Values to Table tool, I start to think that this might not be an easier tool to use than the Extract Values to Point tool. Attached below is a screenshot of the output table. There were two input point features and three input rasters. The output table has 6 (i.e., 2 x 3) records. You cannot join the table to the point featureclass to get what we want. We probably should use the Extract Values to Point tool.

[jdduh]

Please use the Extract Multi Values to Points tool. If you don't feel like to alter the input station featureclasses, then you can make a copy of the input featureclasses and use them to extract raster values.

[lbross]

I had the Extract Values to Table gp tool working well even with that unusual table structure, but if it is unreliable with different extents then it is best not to use it. From a productivity perspective, this means that the work I did last week is wasted, but that's how it goes sometimes. Your mention of the Extract Multi Values to Points tool reminded me that we use this tool for the elevation precipitation correlation chart to extract the precipitation and aspect direction for the sites. This function creates a single feature class with both snotel and snow course sites and appends the precipitation and aspect direction to the feature class. We can add the slope and aspect degrees to this feature class and use it for both elevation precip and the sites information on the report. This means we could add the precipitation value to the report if we want. The elevation precipitation correlation chart takes a long time to generate and this approach would distribute the processing time through some other functions. Also the old code for the sites table on the report did everything twice for snotel and snow course. It would only have one feature class with this new design. Here is a rough outline:

1. When sites layers are clipped, create a combined sites table called merged_sites
2. Add fields for BA_PRECIP, BA_ASPECT, BA_DIRECT, BA_SLOPE
3. Extract Values to points for precipitation, aspect, and slope to populate these fields
4. Use add Join tool for each extract layer
5. Use Calculate Field gp tool to update merged_sites
6. Use Remove join tool after merged_sites is updated
7. If aspzones exists, extract aspect direction when sites are clipped; Recalculate this when generating aspect layer
8. Recalculate and update BA_precip when precip sites clipped
9. Update Elevation Precipitation algorithm to use merged_sites instead of prec_stel with edits to ensure that BA_PRECIP and BA_DIRECT are populated

[jdduh]

Sounds good. Make sure that the table in the report has clear/descriptive headings. If there is a design limitation, then we can add a text box on the report page displaying the detail column descriptions and data units.

[lbross]

This is my most recent iteration. The updated version will look the same but the data will be generated differently. Let me know if you have any changes or want me to add PRISM data. sites_table_8.pdf

[lbross]

Again, I thought I was almost done with this, but it is not working the same on Pro 2.8.1 on CS3. I am using the Data Management Merge tool to merge the Snotel and Snow Course layers into a single layer with all sites. When I run in on 2.6.3 it is fine, but on 2.8.1 it doesn't copy any of the attributes for the second layer (snow course) to the new feature class. My guess is that it is in the field mapping, but I'm not sure where the problem is. I don't want to merge any attributes between the two classes, just to combine all the records. It works on 2.6.3 without any specified field mapping. Any thoughts or maybe a different tool I could use for this purpose?

[jdduh]

I believe this is a bug in 2.8.1. The only way to "fix" it for now is to set a value in the mergeRule attribute explicitly. It doesn't matter what value you set to which attribute field. Once the code "touches" the merge tool's parameters, the tool seems to work correctly.

[jdduh]

The pdf example looks good. Please change the "Descr" heading text to "Aspect Descr".

[lbross]

Thanks for the info about the bug :-( I just got it working with the Append tool so will leave that in place for now. I will update the Descr column. Updated: It looks like I already changed Descr to 'Direction' after generating the example. Is that okay? Trying to keep it short so that the table looks good.

[lbross]

Another quirky problem with 2.8.1. This time with the sample tool. When I try to run it with these parameters on CS3, I get an ERROR 010555: Internal error in Sample: %s.

Input rasters D:\AOIs\2020_NWCC_Basin_Analysis_AOIs\UCO\Yampa\elk_r_nr_milner_09032010\analysis.gdb\aspzoneprec; D:\AOIs\2020_NWCC_Basin_Analysis_AOIs\UCO\Yampa\elk_r_nr_milner_09032010\prism.gdb\Annual; D:\AOIs\2020_NWCC_Basin_Analysis_AOIs\UCO\Yampa\elk_r_nr_milner_09032010\analysis.gdb\precmeanelev

Input location raster D:\AOIs\2020_NWCC_Basin_Analysis_AOIs\UCO\Yampa\elk_r_nr_milner_09032010\prism.gdb\Annual

Output table D:\AOIs\2020_NWCC_Basin_Analysis_AOIs\UCO\Yampa\elk_r_nr_milner_09032010\analysis.gdb\precmeanelev_tbl

"NEAREST"

It looks like the extents are slightly different for a couple of the rasters but this is also the case in another sample AOI and the tool runs fine. I don't think I'm a fan of 2.8.1. It looks like there is a new version. Should I upgrade CS3?

[jdduh]

It needs admin privilege to upgrade. I will perform the upgrade later today. Let me try the sample tool on my 2.8.1 and later on 2.8.2.

[jdduh]

I was able to run the sample tool without any problem on 2.8.1. Did you set the unique ID field value?

Sample

Parameters

Input rasters C:\NWCC\AOIs\Little_Wood_R_ab_High_Five_Ck_AOI\analysis.gdb\aspzoneprec;C:\NWCC\AOIs\Little_Wood_R_ab_High_Five_Ck_AOI\analysis.gdb\precmeanelev;C:\NWCC\AOIs\Little_Wood_R_ab_High_Five_Ck_AOI\prism.gdb\Annual Input location raster or features C:\NWCC\AOIs\Little_Wood_R_ab_High_Five_Ck_AOI\prism.gdb\Annual Output table or feature class C:\workspace\workspace default.gdb\sample_output2 Resampling technique NEAREST Unique ID field VALUE Process as multidimensional CURRENT_SLICE Acquisition information of location data
Statistics type
Percentile value
Buffer distance field or value
Column-wise layout ROW_WISE Generate feature class TABLE

Messages

Start Time: Monday, August 16, 2021 4:07:27 PM Spatial reference wkid and wkt for sample table sample_output2 are [0, PROJCS["NAD_1983_Albers",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Albers"],PARAMETER["False_Easting",0.0],PARAMETER["False_Northing",0.0],PARAMETER["central_meridian",-96.0],PARAMETER["Standard_Parallel_1",29.5],PARAMETER["Standard_Parallel_2",45.5],PARAMETER["latitude_of_origin",23.0],UNIT["Meter",1.0]]]. Succeeded at Monday, August 16, 2021 4:07:30 PM (Elapsed Time: 2.58 seconds)

[lbross]

Thanks for taking a look at this. I am perplexed. I copied the 3 layers into a new .gdb and the Sample tool ran fine. It also ran fine with a different AOI. I ran the tool interactively with the original layers and it completed successfully. I just tried running the full process again and got the internal error :-( I see that you did not update ArcGIS Pro. I can do it because I have the admin information, but I don't know if it will help. I have a couple more things to test.

Updated: Turns out it didn't really work in the new .gdb. I was using the wrong aspect layer. This leads me to believe that the problematic layer is aspzoneprec because when I include it, I get the failure. I ran it on 2.6.3 and it completed successfully. I uploaded the .gdb to basins. It is in the "For Geoffrey" folder. If it fails on your 2.8.1 then upgrading CS3 to 2.8.2 is the next logical step. Unless you see something wrong with the data.

[jdduh]

Got the same error message using your data on Pro 2.8.2. I will let you know if I figure out the problem with the gdb that you provided.

Sample

Parameters

Input rasters C:\workspace\aoitest\sample.gdb\aspzoneprec;C:\workspace\aoitest\sample.gdb\Annual;C:\workspace\aoitest\sample.gdb\precmeanelev Input location raster or features C:\workspace\aoitest\sample.gdb\Annual Output table or feature class C:\workspace\aoitest\sample.gdb\precmeanelev_tbl2 Resampling technique NEAREST Unique ID field VALUE Process as multidimensional CURRENT_SLICE Acquisition information of location data
Statistics type
Percentile value
Buffer distance field or value
Column-wise layout ROW_WISE Generate feature class TABLE

Messages

Start Time: Wednesday, August 18, 2021 2:40:18 PM ERROR 010555: Internal error in Sample: %s. Failed to execute (Sample). Failed at Wednesday, August 18, 2021 2:40:20 PM (Elapsed Time: 2.16 seconds)

[lbross]

I feel better. At least now we have a sample we can provide to ESRI

[jdduh]

Add a new field to the combined snotel and snow course layer and assign a sequential number to each record. Display the snotel and snow course sites on all the maps in the report and label the sites with the sequential number. Add the following text to all the maps: "See the Active Sites table for individual SNOTEL and Snow Course site descriptions." Make sure that the Active Site table also shows the site number.

[lbross]

I understand the concept and will think through the best way to do this. Do we want the symbology for the 2 sites layers to be the same as it currently is (different for snotel vs snow course)? It may be easier to add the id to the source layers that we are already displaying with the maps. Do we want to remove lat/long from the sites table? The reason for adding it originally was to provide a visual link from the map to the sites but if we are using an id for that now, it may make sense to save the space.

[jdduh]

If the sites come from two sites layers, then probably we can add a prefix to the sequential numbers to make it easier to generate the numbers. For SNOTEL sites, we can add ST. Use SC for snow course sites. So the ID would look like ST1, ST2, SC1, SC2, etc. This approach allows the use of existing symbology. Please keep the lat/long data in the table.

[lbross]

OK. We will continue to use the original layers and symbology with the maps. We will need to add the sites layers to some of the maps as they don't all have them. I can use a prefix. We could append the prefix to the layer objectid to generate the id. It's also not difficult to manage the id in C# between the 2 layers so that it is sequential with no duplicates. Whichever you prefer. Also the id's will be regenerated each time the site layers are clipped so they will always be consecutive.

[jdduh]

I prefer just the ID numbers without any prefix so that the labels won't obscure too much of the map and it's easier for map readers to look up from the site table.

[jdduh]

Per the sample tool, ESRI has confirmed that it's a bug. The temporary workaround is to activate the "Create Feature Class" option when executing the tool. That seems to solve the problem.

[lbross]

Switched to the feature class option. Updated code that generates the precipitation correlation feature class and that creates the chart from the feature class. Confirmed that the fixed worked for the elk_r_nr_milner aoi. Uploaded new watershed report to the portal

[jdduh]

The table and maps look good. Some of the station numbers on the maps are obscured. Please force the placement of all labels. See this page for more information: https://pro.arcgis.com/en/pro-app/2.7/help/mapping/text/force-the-placement-of-all-labels-in-a-label-class.htm

[lbross]

Here is what the map looks like with the forced label placement. It seems harder to read. Please let me know that you want to keep this NeverRemoveLabel setting. label_test.pdf

[jdduh]

It seems that Maplex didn't attempt to resolve the placement conflicts. I wonder if the conflict avoidance only happens among features from the same feature class. SNOTEL and snow course points are stored in two separate files. Is that correct? If that's the case we might need to merge them before labeling. Another approach is to place SNOTEL and snow course labels at different positions. Right now they are all placed on the upper-right corner. We can move the SNOTEL label to, say, upper left corner.

[lbross]

I tried your second approach and it seems to work well. Merging the sites into a single file would require much more rework. Please let me know what you think of the new sample. label_test.pdf

[jdduh]

How about placing the labels in similar (as oppose to opposite) directions? Upper left and upper right or lower left and lower right? I know this doesn't solve all possible placement conflicts but let's give it a try when we test the batch tool on multiple AOIs.

[lbross]

Sounds good. Will do.

FYI there are a couple of snotel sites almost on top of each other in my version of the Animas AOI. When I use the forced label placement, it works the same as it did with multiple layers. That is Maplex doesn't try to resolve the conflict, it just places them on top of each other like it did in the earlier sample. So merging the sites into a single file won't resolve this problem :-(

[lbross]

1. Update batch tool settings. Set SnotelBufferDistance to 5 and SnotelBufferUnits to Kilometers. Setting the buffer distance this large for the sites layers is proving to be problematic. We calculate the elevation for the sites by using the ExtractValuesToPoints tool with the DEM. With this large buffer, we are clipping sites outside of the DEM extent so we can't calculate their elevation. Thus we can't work with them in any of the downstream processes. We also can't get any of the other surfaces information like slope or aspect.

2. What should the default buffer distance be if nothing is entered in the Clip Snotel/Snow Course tool. We have been defaulting to the AOI buffer, but I wonder if it should be the batch tool settings = 5 kilometers?

3. Rename “Id” column to “Site #” in table

4. Can we rename the 'Snow Course Sites Represented Area' layer to 'Snow Courses Represented Area' or 'Snow Course Sites Rep Area' so that the name is short enough to appear in the legend without being cut-off?

[lbross]

1. @jdduh to email Gus
2. Use batch tool buffer settings for both sites and PRISM buffer layer
3. No questions
4. 'SNOTEL Rep Area' and 'Snow Courses Rep Area'

[jdduh]

Per item#1, we could clip a separate copy of the DEM to the AOI with the required buffer distance (5 km) and calculate SLOPE on it. These additional "surface" layers will be saved to AOI's report layer folder/GDB. The original DEM and SLOPE won't be altered.

[lbross]

We would need to do ASPECT too. Do we need to do any processing after clipping the DEM to make it a "filled" DEM? We do all of our analysis on the filled DEM. I'm not sure how this is will work with the charts, especially the Precipitation Correlation chart. But I guess we can try.

[jdduh]

Yes, we need ASPECT. There is no need to fill the DEM. We shouldn't perform the analysis on the filled DEM, but I think the difference of the results is negligible. Filled DEM is not the real DEM and should only be used for watershed delineation purpose.

We can limit the precipitation correlation calculation to the extent of the PRISM raster. Have we set the extent explicitly in these analyses?

[lbross]

Just finished extracting the PRISM values for the sites outside the PRISM buffer. It seems to flow pretty good and they are included in the elevation precipitation calculation. At this point it would be more work to exclude them, but I can look into that if we need to.

[jdduh]

Truckee R at Farad 10346000:CA:USGS has too many sites to fit on one page. I believe we will encounter more of these when generating reports for large AOIs. This AOI has a total of 46 sites/stations. The site table can only fit 35 sites on one page. We need to create an appendix if an AOI has more than 35 sites and leave the site page blank (with the text directing the readers to the appendix). The appendix will be added to the end of the report.

[jdduh]

This is text for the active sites table page when there are more than 35 sites in the AOI.

"ACTIVE SITES TABLE This watershed has more than 35 sites. See Appendix for the complete list of sites. THIS PAGE INTENTIONALLY LEFT BLANK"

Do we have a similar blank page when there is no site in the AOI?

[lbross]

This is done. I overwrote a sample report on the FTP server called 10217000_UT_USGS_Watershed-Report.pdf for your review. I dropped the maximum number of sites to 25 because sometimes the names of the sites wrap, so that a table with 35 sites is still too long for 1 page. The page break is not always where we might want it to be, but because this is html I can't control that. We did not have a similar blank page when there are no sites in the AOI. We do now :-)

[jdduh]

Looks good. This particular report doesn't have maps on the multiple-map pages. Just want to make sure that's because you turned the option off for debugging purpose.

[lbross]

You are correct. The multiple-map pages were disabled for debugging. Glad it looks good.

[jdduh]

Obscured station number (see the image below). I believe that we talked about this issue but NWCC brought it up again. This could be a design challenge. Do you have any thought? We could expand the map frame to cover a larger area, but all the maps will look smaller.

[lbross]

We inherited the MapFrame dimensions from BAGIS V3. We set the upper right and lower left corners when we create the map frame. The current values are: LL 1.0, 3.0 UR 7.5, 10.0

It looks like we may have some space to give horizontally, but not vertically due to the other map elements.

The envelope for the map comes from extent of the unbuffered AOI vector. There is a configurable expansion factor that can be set. It is currently set to 1.1 but could be changed.

Let me know what you think.

[jdduh]

We could make the MapFrame a little bigger (e.g., 0.7, 2.7 for LL and 7.8, 10.3 for UR) and set the expansion factor to, say, 1.3. This also assumes that the background of the MapFrame is transparent and won't obscure other map elements. Please give it a try and see how the map looks (and if it shows all the station numbers for the San Miguel River AOI). We might need to tweak the frame size and expansion factor.

[lbross]

Just a quick update. I've spent most of my time today on this and am puzzled. When I change any of the dimensions for the envelope, even just 0.5 inch, the maps no longer load in the center of the map frame. The vertical positioning seems wrong. ArcGIS must be doing some calculation with the coordinates that I can't figure out. The layout size is set to 8.5 x 11 because we are doing letter page size. And when I look at the layout dimensions in the Pro Client, the numbers that are reported don't reflect the xMin xMax etc. I will keep plugging away next week, but wanted to see if you had any ideas?