Represented precip specification

[lbross]

The design spec says the output graphics and tables are stored in the Basin Analysis Excel spreadsheet. When the spreadsheet is initially created, it isn't saved anywhere. The user has to save it. So ... do we ask the user to provide the location of the spreadsheet so we can append? Or is this integrated with the creation of the Basin Analysis Excel spreadsheet? If integrated, do we give the user the option of appending it? Or is it automatically created when the user clicks the 'Tables' button on the 'Map Settings' form?

[jdduh]

Sorry, I meant "put the output in the Excel spreadsheet." Let users determine if they want to save the spreadsheet. The current workflow works well for NWCC. The tables and maps are created on-the-fly. We need to generate (and keep) all the GIS data that are required for the precip analysis when, for example, users click on "3. Apply". We might need to add a checkbox asking if the precip representation analysis will be performed when users click on the "Tables" button. The default value is "checked." You can use the Data Status list (on the upper right corner) to keep track of the required layers. If any layer is missing, then the Tables and Maps buttons are disabled.

[lbross]

I plan to integrate the represented precip functionality with the existing 'Tables' functionality. I don't suppose you have a list of all the formats you applied to the sample chart? And would like to have applied to the BAGIS chart?

[jdduh]

The original formats were provided by NWCC. We will need to come up with our own design for these new charts/tables. First off, we can name the new sheets "Precip-DEMElev", "Precip-SiteElev", and "Precip-Elev Charts." You probably can use the standard column headings for the tables and symbols for the charts (see the mock-up excel file that I sent you).

[lbross]

The precip analysis doesn't take that much longer than the other maps analysis so let's not do a checkbox for now. We can add later.

Question on Data Status list. We generate 3 files for this function. The aggregate tool output is a raster. I have added this to the Data Status list. The sample tool output is a table and the extract values to points tool output is a vector. Does it make sense to add them to the data status list since they are not rasters? It is easy for me to check for their existence behind the scenes in code.

Currently the only qualifier to enable the tables button is that all the layers in the Data Status list are there and that Excel is available. I could add the check for table and raster layer here but if one of them is missing, the user won't know why the Tables button isn't enabled. MessageBox or ...?

Is the precip layer always in inches? I thought it could be inches or mm but the frmGenerateMaps seems to only support inches.

I'm attaching my current excel output with the distributed precip so you can take a look. I know I still need to add the other 2 series and the R-squared values but let me know what you think so far. BagisSample2.xlsx

[lbross]

Do we want to include both Snotel and Snow Course data in this chart? Do we combine Snotel And Snow Course data into one data series? Or will we have 3 data series on this chart with 3 trendlines?

Updated: Can I not use the existing SNOTEL and Snow Course sheets in the BAGIS tables worksheet? This already has the elevation making this new chart consistent with the other charts on the sheet. I can extract the PRISM values to a new column on the right-hand side.

Updated again: It seems there is no key so I have no way to append the prism data. Boo :-(

Updated again: Okay to re-use existing BA_ELEV value off the snotel and snow course files? Otherwise, I'll have to run the extract multi-values to points tool twice and somehow combine the output.

[lbross]

What do we do if the partitioning raster is a different resolution from PRISM (likely)? The aggregate tool isn't a good choice because none of the aggregation strategies are appropriate for a discrete raster (Sum, Min, Max, Mean, or Median), Should we plan to do a resample using the majority function?

How do I combine the partition raster values with the values generated by the Sample tool to create the Excel table. Sample tool doesn't have good aggregation strategies for discrete data.

Also do we let the users choose the raster value(s) that should be filtered? > 1 value? Should the values they choose by included or excluded from the calculation?

[jdduh]

It's fine to reuse BA_ELEV values. Yes, please combine the SNOTEL and snow course layers and add an attribute field (e.g., "TYPE") to indicate if a site is a SNOTEL, snow course, or other type. NWCC can use a filter to separate different types of sites.

I have noted in the Google slide (BAGIS V3.1 Site Scenario Tool) of the "resampling" methods of the Partition Layer. Use NEAREST for the sampling tool. We need to generate a 3 by 3 majority output for the majority option.

At this point, we don't alter or filter the value of the input partition layer. The filtering can be performed in Excel. Let me know if I mis-interpreted your question.

[lbross]

The Excel API doesn't allow me to move the trendline labels into the right-hand margin. I think internally it restricts the location to a certain number of pixels away from the line. I can set the height but not the left location. See attached image. Thoughts?

[jdduh]

The locations of the legend and regression equations are less important than the color and size of the texts. As long as the texts do not obscure the graphics, any location is acceptable.

[lbross]

If there are no snotel or snow course sites do we still generate the chart with the DEM cells? Or is this an error condition and we warn the user and don't produce the chart?

[jdduh]

If there is no snow monitoring sites, then we don't create the Elev-Precip Sites spreadsheet. If the site number is 1, then Excel cannot generate a trend line nor r-squared value. Please prompt a warning message when the "Tables" button is clicked, but still allow the table/chart be created with partial data.

[lbross]

What resampling strategy should be used for the aspect layer to resample to PRISM cell size ?

[jdduh]

The recoded/reclassifed aspect layer has the same spatial resolution as the dem. No resampling is required. When extracting values to the dem cell locations, only NEAREST can be used because the recoded aspect layer is nominal data (discrete number) and must not be averaged.

[lbross]

Doesn't it need to be resampled to cell prism size? We use the aggregate tool to resample the dem to prism size but, as you pointed out, this doesn't work because aspect is discrete.

[jdduh]

Sorry, I lied! I have added a new slide (#8, "AOI Partition (Additional Attribute) Layer Processing") to the Google slide document. In short, we need to use Focal Statistics tool with the Majority tool. See the slide for more information. You might want to see if setting the output raster cell size to DEM or PRISM makes any difference in the final result. If the results are the same, then use the PRISM cell size in the Focal Statistics.

[lbross]

Got it. I also plan to reclass the aspect raster to the directions (1, 4, 8, etc.), according to the logic used to create aspzones, prior to running the focal statistics tool.

[lbross]

I tried to implement the focal statistics for aspect this morning. It takes a REALLY long time to run with an aspect with 22m resolution and PRISM 935m resolution. It's been going for almost an hour now for the Animas AOI. Is this what you would expect? Note that I set the snap raster to PRISM layer.

[jdduh]

Make sure that you run the tool on an integer layer, not directly on the aspect layer. The aspect layer needs to be reclassified into 4, 8, or 16 directions. I did a test on Aminas and it took 1 minute and 5 second. See the result message for the parameters that I used.

[lbross]

Got it working. My biggest error was using cells instead of map for the units. I then had a problem with the sample tool until I realized it was setting cell size to the minimum. The aspect layer is much smaller than the others. When I set it to the prism cell size it worked.

[lbross]

See slide 10 in Site Scenario site requirements. Worked on incorporating status of elev-precip layers into Map Settings UI. Couldn't add them to Data Status list because that only counts the number of layers to know if everything is ready. AOI streams is an exception but you can only have one exception when you are counting. Added disabled checkboxes to the elev-precip correlation groupbox that indicate if the layers are in place. Also use these together with the generate elev checkbox to manage generate and tables buttons. Does this work?

[lbross]

Seeking clarification on slide 8 regarding aspect and partition raster layers. The slide says, "Allow users to pick a “resampling” rule from: “Nearest Neighbor” or “Majority among 3 x 3 neighbors.” Nearest Neighbor is the default." Is this still accurate?

Earlier in this thread it talks of using the Focal Statistics tool with the Majority option. This is how I have currently implemented aspect. Is this correct? And should partition raster be implemented the same way?

[lbross]

Attached is help text for the Elevation-Precipitation Attribute Layer selection screen. Please review and make changes if desired. It is long. I wonder if I should make it a .pdf and launch .pdf from the help button? Partiion.docx

[jdduh]

Looks good. See the attached document for edits. We need the message format to support hyperlink, but I don't like to use another software (e.g., Acrobat Reader) that users might not have installed on their computer. If we don't want to support hyperlinks in the message text, then we can put something like "Search for "Filtering Data in Excel" on the web to learn more about partitioning data in Excel."

Partiion_GD.docx

[jdduh]

NWCC has an enhancement request on this tool. They like to have an additional option to aggregate the elev and precip data. The current, default aggregation is performed on PRISM cells. Please add a new aggregation option that is based on an input raster zone map. The new aggregation option requires both PRISM and DEM be resampled. I will put the detail descriptions on the "BAGIS V3.1 Site Scenario Tool" Google slide.

[lbross]

Could I also get a sample of an "input raster zone map"? Or some instructions on how to create one? Would we still have a point for every cell? Or would it be a point for every zone on the graph?

[lbross]

I will implement the Help architecture from BAGIS-H in BAGIS V2 which doesn't support hyperlinks. I will add the comment you suggested. Please send updated help text if it needs to change based on NWCC enhancement request.

[jdduh]

It would be one point for every zone.

[lbross]

Was this their original intent for the partition raster? Or is it in addition to the partition raster? The partition raster itself as currently implemented doesn't seem that useful to me since we already have aspect. We could re-use the UI in a different manner?

[jdduh]

You can create a HRU layer for Animas AOI and import (use) it as a "zonal" layer. If the layer has only 30 HRU zones, then the output AOI data will have only 30 records (i.e., 30 points on the scatterplot). This is different from the additional attribute layer function. The attribute is for examining how different attribute values (e.g., aspect) affect the correlation. The aggregation is for examining how different spatial aggregation of precip/dem data could affect the correlation. PRISM cell provides a systematic aggregation scheme, whereas the zone layers allow for user-defined aggregation scheme.

[lbross]

Does the use of the "zonal" layer ONLY affect the 'All DEM cells' data series in the Elev-Precip Chart Would need to change the series name, of course. Each point represents the mean precip/elev for a zone? How are we handling differing resolutions? Does the zonal statistics tool take care of this for us? Do we need to update the Elev-Precip AOI table to include the zone number? Do we need to update the Elev-Precip Sites table to show which zone the site is in? How do we handle aspect and partition layers since these are discrete? Don't include? Majority rule?

It seems like we might need a separate processing path for this option since much of the processing will be different.

[jdduh]

Zonal layer only affects the All DEM cell data series. How about changing 'All DEM cells' to just 'AOI'? Yes, use zonal mean (zonal statistics as table) to get precip/elev values. It seems the tool can perform resampling before zonal stats. Make sure that you set the raster processing Cell Size to "minimum of inputs." Yes, it would be useful to include zone ID/number in the output spreadsheet, but I like to see the spreadsheet to have a consistent format whether a zonal layer is used or not. Interesting idea to show the zone number the site is in. This information can be derived using the extract value by point tool. Yes, please add the information to the output. David might be interested in seeing the information. Again, I prefer some consistency in the spreadsheet layout/format. Use the majority rule in zonal stats tool. You will need to have two separate processes for adding the new zonal layer feature.

[lbross]

We have the problem, again, of retaining the zone layer information between sessions because it's not part of the log file. We can't use the same approach as partition raster because the layer doesn't live in the analysis.gdb folder. Do you think I should create a separate .xml configuration file to store the configuration for this part of the tool? Any other ideas?

I understand most of this. I updated slides 10 and 12 with detail in green. Please review, fix any mistakes, and change to orange when you approve.

As for appending the zone to the tables, it will be like the partition raster. If it's not used, the column won't be there. I will always put the columns in the same order. Does that work?

[jdduh]

How about 'importing' the zone layer before processing it so that it becomes permanent to the AOI? The zone layer will have a predetermined file name once it's imported to the AOI. If this is acceptable, then you can use the same logic for handling the attribute layer.

The solution dealing with the output spreadsheet format sounds good.

[jdduh]

I have updated the slides and changed the reviewed texts color to orange.

[lbross]

Regarding letting the user select the zone field for the zonal statistics as table tool; Zonal statistics as table DOES support using different attributes BUT this gives us something else to keep track of between sessions. If I 'import' the layer before processing, I could copy the values from the selected zone field of the original raster to the VALUE field on the raster and always use that field. Extra coding and processing time. We could also always just use the VALUE field which is consistent with how we use the focal statistics tool when aggregating the aspect/partition rasters.

[jdduh]

Make VALUE the only acceptable field for the zone layer. This limitation should be made aware to the user. We can consider putting the 'Help' button on the main basin analysis dialog window (i.e., the map settings dialog window) instead of the attribute layer dialog window. The caption could be something like 'About Elev-Precip Analysis'. I will update the help docx file.

[lbross]

Let's put another 'Help' button on the zone layer selection form that explains how the zone layer works. It is different than the regular processing. Since I implemented the framework for help in BAGIS v3, it is easy to add. So, yes, please update the help docx file but make it a separate section. The map settings form is already quite busy.

[lbross]

Please take a look at slide 13 to review the updated map settings form. The checkboxes are still there indicating the presence of the layers. I plan to switch them eventually but for now they are easier for development. The read-only properties of the zone aggregate layer are dependent on the radio button.

[jdduh]

Sounds good.

[jdduh]

Partiion_Version3.docx

[jdduh]

The updated map settings form looks fine.

[lbross]

Permutations to validate:

1. Aggregate by PRISM cells, no attribute layer
2. Aggregate by PRISM cells with attribute layer
3. Aggregate by zones, no attribute layer
4. Aggregate by zones with attribute layer