Parameters from Layers Tool

[jdduh]

Add the new tool to the BAGIS-P AOI Parameterization pull-down menu.

Main purpose:

The tool allows users to set HRU parameter values by finding the dominant raster values in each HRU zones and assign the output parameter values based on the values in a user-specified look-up table.

[jdduh]

Here is a mock-up. The main computation is based on the zonalstats majority type. The output values are from the lookup table (borrowed from BAGIS-H reclass discrete rule).

[jdduh]

Updates to the export dialog window:

1. Add a label/text box saying: There are X pre-calculated parameters in the selected HRU.
2. Add a checkbox saying: Overwrite BAGIS-P parameters using pre-calculated parameters when duplicates are detected.
3. Add a label saying: "Please use BAGIS-P Parameters from Layers Tool to manage pre-calculated HRU paratmeters."

[lbross]

Notes/questions on new Parameter from Layers tool

1. Select an AOI
2. Select an HRU
3. Use reclassified raster to set HRU param values. Select another HRU in the AOI or a raster in the layers.gdb. Do users get to choose the field? Or is it always Value? The table implies this is a batch interface. Can they process multiple layer/parameter name combinations? Calculate the zonal statistics majority for the reclass field for each HRU and append to grid_zones_v.
4. List pre-calculated parameters in HRU grid_zonesv. What to list? Naming convention? Do they start with hru? Do we list any columns not created for other purposes in BAGIS-P?
5. 'Delete selected parameters button': Does this delete columns from grid_zones_v?
6. What is the purpose of Field name(s) textbox in lower left-hand corner? How is this populated?
7. Do we need an interface for user to view these calculated parameter values? Either in table or map?
8. Should we rename 'View PRMS parameter names' button to 'View parameter list' so that it has the same name as a button that does the same thing on the export dialog window? Or should we rename the button on the export dialog windo to view PRMS parameter names?

[jdduh]

1. Ok
2. Ok
3. Clipped rasters sometimes contain additional fields. Please implement the tool similar to what you did for BAGIS-H's reclass tool. Set the default to VALUE. No, the tool perform the computing one parameter a time. Do we put SubAOIID to the grid_zones_v as well?
4. Just list the names of the parameter attribute fields. Please filter out the "standard" non-parameter fields. The field name should be the parameter name that user provided when the parameter was calculated. We do allow users to type whatever name they want, but PRMS will use only the parameters it recognizes (i.e., the parameters listed in the bagis_parameters.txt file).
5. Yes.
6. This is where users key in the parameter name (i.e., hru_deplcrv or jh_coef). Please rename the field name(s) to Output parameter name.
7. Good idea! Can this be done with the current parameter viewer (with minor modifications)?
8. Please rename the button to "View parameter list"

[lbross]

Here is summary of how subAOIID functionality is implemented: https://sites.google.com/site/nwccspatialservices/developers-references/bagis-p/bagis-p-screens#TOC-SubAOI-ID-Tool. The subAOI information is maintained in a separate raster and appended to the output parameters table when parameters are calculated for an hru/profile.

Appending the custom user parameters to grid_zones_v is better because we only have to do it once and they are accessible to all profiles. It means more coding work to change the parameter export but will ultimately be more efficient.

[jdduh]

Ok, thanks! SubAOI is an AOI level information. The parameters discussed in this issue are HRU level data. We can keep them separated unless there is any advantage of storing the SubAOIID in the grid_zones_v layer.

[lbross]

See attached for updated export parameters dialog. Not yet functional. Suggestions?

[jdduh]

Looks good.

[lbross]

First draft of Parameters from layers form. Need clarification on what information should be in the old/new values table underneath the reclass field. Will re-arrange the form a bit once we have settled all the fields.

[jdduh]

Rename old values to "Layer Values", new values to "Parameter Values." Rename Reclass field to "Layer value field"

The layer values column lists all values in the "Layer value field" of the raster and the parameter values column is editable (set its default values to the same as what are in the Layer Values.

[jdduh]

The Parameter Values are transferred to each HRU based on the majority occurrence of the Layer values in the HRU. Only discrete rasters (including HRU) are valid input layers.

[lbross]

Assigning the parameter value is a 2-step process. Correct?

1. Select the majority value for each hru. This will be the layer value
2. Substitute the parameter value from the second grid if the user changed it to be different from the layer value.

Do you want to set a default for the layer type radio button? HRU or raster?

[jdduh]

The process is different from the reclass tool for HRU delineation. Step 1 is correct. There is no 2nd grid in Step 2. Once the majority layer values are determined for the HRUs, replace them with the matching "parameter values" specified in the look-up-table.

Don't reclassify the input layer to generate the 2nd grid. You can just join the look-up-table to the param output table using the input layer value as the key and copy the parameter values directly to the param output table. I describe this approach based on ArcGIS tools/functions. There might be other ways to achieve this with ArcObjects/ VB codes.

Set the default to HRU.

[jdduh]

When I said "param output table," I meant "grid_zones_v."

[lbross]

I mis-typed in my earlier comment. I meant to say, substitute the parameter value from the second column if the user changed it. It sounds like this is what you meant.

I need to set the field type on grid_zones_v for the new column. Will the values always be integers? Or should I copy the column type from the Layer Value field type?

We will also need a length for the field if it is a string.

Where should I record the date that the parameter was last calculated? It would be easiest to just add a date column to grid_v but it is a waste of space because the date is the same for all HRUs. It could be appended to the log.xml that is generated when the HRU is created but this file has been solely managed by BAGIS-H. We could create a new log file for the parameters from layers but this would be more work. It is the cleanest solution, though.

[jdduh]

The layer value field should always be integer, but not the parameter value field (for example, hru_deplcru is an integer, but jh_coef is a floating point number). I don't recall that we have any string parameter values. It probably is fine to use a float field to store integers (i.e., 2 vs. 2.000).

I prefer a new logfile so that the tool won't interfere with BAGIS-H.

[lbross]

Do we want edits for the "parameter values" in the grid? It sounds like the value should be numeric and that decimals are acceptable. Are negative numbers acceptable?

[lbross]

The ArcGIS zonal statistics tool does not support specifying a field on the input raster. It can only calculate the statistics on the "value" field. Can we modify the UI to only show the values from the "value" field? As a workaround, the user could use the reclass tool to reclass the raster using the desired field.

[jdduh]

On the one hand, I like the tool be flexible, but on the other, the tool must be simple and easy to use. Let's just limit the field to VALUE and not allow negative values.

[lbross]

Test floating point value if possible. We WILL allow negative values.

[lbross]

I am running into some tiny hru's (one cell) where ArcMap is not able to calculate a majority value. How shall we handle this? Add a NoData value to the form? Use our own NoData value of -9999? Do we need to warn the user that this happened?

[jdduh]

Can you add a NODATA entry in the look-up table so that user can specify their preferred NODATA value? We can set the default to -99 (or -9999). Please add a warning message on the dialog window saying that "Single cell HRUs will be assigned a NODATA value because the MAJORITY tool does not support single-cell evaluation."

[lbross]

Any suggestions on how to create an input raster with float data types in the value field for testing? I just tried the Reclassify tool and it switched my input to integers. The data type of Value in the output later was Long.

[jdduh]

Try raster calculator. For example, "inraster" * 0.1

[lbross]

Have added the NoData entry to the look-up table. Do we want to save any data to the log file in addition to the parameter name and calculation date? Source layer? Reclass table?

[lbross]

The raster calculator works, but then ArcMap thinks it is a continuous dataset and won't let me build an attribute table or look at it using the Parameter From Layers tool. I hope this tool will still be usable for NWCC. If the parameter values were floating, they could create a raster without the decimal point and then put it back in manually on the reclass table in the Parameter From Layers tool.

[jdduh]

Good idea... How about all the attributes - layer type, name, and the LUT? Could you add a button below the Delete Selected Parameters button with the caption "View Parameter Log"? The button opens up the .xml logfile in a web browser (i.e., in read-only mode).

[jdduh]

Limited to integers is fine. Thanks for checking.

[lbross]

About to add these parameters to the export parameter calculation. Do we ALWAYS include these parameters with the export? There is a checkbox on the export form saying that duplicates will be overwritten. It seems logical to:

1. If overwrite duplicates is not checked then include any layer parameters that are not duplicates
2. If overwrite duplicates is checked, then include them all

We are using the layer parameter log file to determine the existence of layer parameters in grid_zones_v. The parameter names need to be present there and as field names on grid_zones_v.

[lbross]

I am integrating the parameters from this tool into the parameter export process and am rethinking where we store the parameter values. Since we've beefed up the log and are now storing the LUT there, does it make sense to just pull the parameter values from there during export? Accessing the xml log file is a lot faster than having to open the data layer and read the attribute table. Don't know if there are any archiving ramifications in eBagis.

Scratch this. After thinking it through, I realized that we don't store the parameter value for each HRU in the log.

[jdduh]

Exporting pre-calculated parameters. Please make the following edits on the GUI texts.

1. Change "There are 2 pre-calculated..." to "There are 2 sets of pre-calculated..."
2. Change "Overwrite BAGIS-P parameters using..." to "Export pre-calculated nhru parameters (duplicate BAGIS-P output parameters will be overwritten)"

This means that the pre-calculated parameters are exported only when the checkbox is selected. When the checkbox is selected, all pre-calculated parameters are exported. The pre-calculated values overwrite the BAGIS-P output values if duplicate parameter names are detected.

I don't expect the log file store hru parameter values. I believe you still have to access the grid_zone_v attribute table to get the hru parameter values. The log file only stores the meta data of the calculation, but not the parameter values. The additional nmonth parameters (PE_Obs and SR_Obs) are stored in the log file because there is no associated layer attribute table. Let me know if I'm wrong.

[lbross]

You are correct about the log file not storing hru parameter values. There are also some spatial parameters imported from the template that are not in the list of BAGIS-P parameters. My current design overwrites these as well if the box is checked and the pre-calculated parameters have the same name. Hopefully this is okay...

[lbross]

Initial release of BAGIS-P with tool posted: https://github.com/PSU-CSAR/vb-bagis-p/releases/tag/1.9.4

[lbross]

Add new button to form: View Parameter Values on map. Re-use code from ParameterViewer to add map to display. No labels. Use random color/unique values symbology.