Auto site selection specs clarification

[jdduh]

1. Has there been an update to the Precipitation Distribution chart shown on slide 4? This chart is in the BAGIS manual but when I produce the charts using BAGIS, the top axis is now named 'Area-Elevation, Precipitation and Site Distribution' and displays the % contribution by elevation zone rather than the inches of precipitation.

2. I propose modifying the UI to only work one scenario at a time. Then we can add a tool that allows users to compare 2 scenarios. We may want to do some restructuring of how the metadata is stored but it would make the UI cleaner and the tool(s) more powerful.

3. Algorithm for a single site: a. Would we have a single polygon for each non-represented area/range? How to handle non-contiguous areas? b. Understand the multiple ring buffer tool and how to use a negative # for the distance to get inner buffers. Don't understand how you would use the Euclidean distance tool with this. Why not just get the centroid of the innermost buffer?

4. For multiple sites: How do you divide the non-represented polygon to create multiple scenarios with the multi-ring buffer? It would seem you would get the same answer every time.

5. Does the BAGIS maps process keep the additive precipitation layer that it used to create preczone?

[jdduh]

1. Yes, charts captions and titles have been updated. The user manual needs to be updated for BAGIS V3.
2. Keep the two scenarios design for now unless there is an imperative reason to change. The two scenarios design allows for direct comparison between the actual and pseudo representations. Auto site tool will be added to the UI.
3. Try a proof-of-concept approach based only on non-represented area. Need input from NWCC for further refinement.
4. Table for now.
5. No, but will consider caching the additive precipitation layer in future release of BAGIS.

[lbross]

Algorithm for a single site: The represented area is stored by our current site scenarios tool as a vector. I can:

1. Use the 'erase' tool to mask out the represented area from the AOI boundary
2. Use the multiple ring buffer tool to get to the innermost of the non-represented area
3. Get the centroid of the innermost buffer to find the psuedo site location

Does this sound like a good approach? If so

1. What version of the AOI boundary should I use?
2. How should I determine what width of the multiple ring buffer tool to use?

[jdduh]

The algorithm is correct and should be applied on aoib_v boundary file. The buffer distance should be negative values (so that the buffers are created inwards). Here is the algorithm for getting the innermost buffer:

0. Set aoib_v as the initial polygon
1. If the area of initial polygon is smaller than a threshold value (e.g., 100,000 square meters), then
   • 1.1 Explode the multipart polygon into single parts
   • 1.2 Find the largest polygon and return its centroid
   • 1.3 Stop
2. Get the shorter of x and y dimensions (extend) of the initial buffer and set the buffer unit (BU) lenght to 1/10 of the shorter dimension.
3. Create multiring buffers using -0.5 x BU, -2 x BU, -3 x BU and -4 x BU as buffer distances
4. If no buffer is created, then BU = BU/2, repeat 3, else set the inner-most buffer as the initial polygon, repeat 1.

Hopefully the algorithm can handle all kinds of cases.

[jdduh]

It seems it's much easier to just convert the vector to raster and use the Euclidean distance tool! Update: you might be able to use the vector as input of the Euclidean distance tool.

[lbross]

Running some tests. Euclidean distance almost works with these settings:

1. Input features: npactual (represented area vector)
2. Mask: aoib_v

The problem is that the furthest distance is always going to be on the border of the AOI which is NOT what we want, I think. Should I

1. Add a line of cells around the AOI border; Create buffer width same as cell size of dem.
2. Merge that with the npactual features
3. Run Euclidean distance

This should get us locations equidistant between the represented area and the border. If this is correct, what should I use for cell size for Euclidean distance? Default for vector is "the shorter of the width or height of the extent of input feature, in the input spatial reference, divided by 250"

[jdduh]

I will check with NWCC on their preferred auto-site location criteria. Distance to AOI boundaries should not be a concern (they might want to pick a site that is shared among multiple AOIs). Elevation range is more of a concern. For example, in low elevation there is not much snow to be monitored.

If computation time is acceptable, then we can use the DEM cell size in the Euclidean distance tool.

[jdduh]

NWCC might direct us back to where we began! Distance to AOI Boundaries is not a concern. They expressed interest in setting an elevation search range AND proximity to, say, road network. We will need to put these two search constraints on the GUI, next to the Auto Pseudo Site(s) button.

[lbross]

I worked on the prototype as we discussed and uploaded the slides to google drive. I shared a copy with you. Hopefully you got an e-mail. The only thing that didn't work as expected was the Combine tool. When I combined all the layers, I got no values of 3. The raster calculator did work. Now I'm stuck. I'm not sure how to use the Euclidean distance tool with the combined layer. Any suggestions? I have all of the participating layers in a file .gdb that I can post on basins if you want to take a look.

[lbross]

I tried a new approach to generate a vector for the Euclidean Distance tool. I unioned the existing sites, undesired elevation, and undesired proximity layers. Then ran Euclidean distance against that using aoi_v as the analysis mask and got a decent result. I'm not sure of the best way to select the highest value pixels from the Euclidean distance layer. I updated the site scenario prototype .ppt on google drive. The alternate approach starts about halfway through the slides. Take a look, run it past NWCC, let me know what next steps should be to actually locate the sites.

[jdduh]

The original raster approach issue probably is related to NODATA cells. You can reclassify NODATA cell to 0 or some other number to correct the problem. The vector output result looks promising. I'm not sure which approach is the most efficient (maybe it all depends on the size of the AOI and terrain complexity). For single pseudo site option, you can query the statistics of the raster to get the maximum value and use a conditional statement (CON) to select the pixels that have the max value. Then, convert these selected pixels to polygons and find the centroids of these polygons. Additional rules need to be developed (e.g., select the largest polygon) if there is more than one polygon/centriod.

[lbross]

I'm not sure what the issue is with NODATA cells. It's counter-intuitive but Euclidean distance calculates the distance from cells with values to those without. So, we need to set the desired area for site locations to NODATA. With the vector approach this means the desired area is NOT part of a polygon. I think vector is most efficient because represented area and the roads layers are already in vector format.

I finished the last few steps of the prototype according to your instructions. The hardest part was setting the pixels where we don't want the sites to null so we can use the raster to point tool and not get a bunch of points. Euclidean distance returns a float value so adjacent pixels likely won't have the same value. I guess we could randomly pick one if they did.

Which AOI boundary should be used? I used aoi_v in the prototype.

I updated the prototype .ppt on the google drive. Let me know if you actually want me to start coding this. It will be a lot of hours just to do the single site. We may want to meet again too before I start.

[lbross]

Added a sample form on p. 4 of the site scenario tool spec slides.

[lbross]

Are the the elevation and/or spatial proximity layers optional?

[jdduh]

Farthest criterion and non-represented are required rules. Elevation, precipitation, and additional spatial constraints (i.e., proximity) are optional.

[lbross]

I have updated the Site Scenario Prototype on google drive if you want to take a look. I have a couple of specific questions:

1. Which AOI boundary should be used? Let the user choose? I used aoi_v in the prototype.
2. When the process completes, we have a feature class with one point in it. We can pop a messageBox informing the user that the process is complete; that they can view the map by clicking a button. With the current site scenario tool, pseudo-sites live in their own feature class. We could automatically add the new pseudo-site to this layer as part of the process. You probably want some way for the user to accept/decline the new pseudo-site but I don't know how to do this interacting with the map. The user always has the option to delete the pseudo-site through the site scenario tool. I think the code is already written to do this.

[lbross]

See slides 3 and 4 for updated GUI

[jdduh]

1. Please use aoib_v as the analysis boundary.
2. Notifying that a pseudo site is created is sufficient. Users need to manage the pseudo sites using the site scenario tool. Will sites generated in multiple runs of the auto-pseudo-site tool be merged into one featureclass, or they are in separate featureclasses?

[lbross]

Add the point generated by the auto pseudo-site tool to the existing (maybe) pseudo_sites feature class.

[lbross]

Note: In current processing for the Site Scenario tool, if the user uses the pseudo-site tool, the new site is added to the Existing Sites list (top) and it isn't selected to be part of the analysis. We need to put some more thought into how the workflow will work for iteratively, automatically, adding pseudo-sites.

[jdduh]

The auto pseudo site tool uses additional criteria/constraints in site-representation analysis. We need a way to view the settings of each pseudo site. Maybe through an xml file? The information can also be stored in the attribute table of the pseudo_sites featureclass.

[lbross]

It sounds like you want to see the parameters for each psuedo-site, not just the most recent? We could add this data on to the end of the report that is currently accessible from the site scenario tool. Or we could add a report button to the auto pseudo site tool that only reports on the pseudo sites. How shall we represent sites that are added manually using the button on the site scenario tool? An xml file is the better way to do this as the structure is already in place.

[jdduh]

The report only shows the sites that are used in the calculation. How about reuse the auto pesudo site form to display the settings when a record in the Scenario 1 list is double-clicked? If a manual pseudo site is double-clicked, then just prompt a message box saying "The pseudo site location was digitized by the user."

[lbross]

So load the auto-pseudo site form with the settings that were used to create the pseudo-site. Do we want the form in read-only mode? Or should the users be able to update it and regenerate another pseudo-site? The maps button would need to be disabled either always (for consistency) or if the timestamp of the represented area was later than the pseudo-sites.

[jdduh]

Make it read-only and hide all the buttons except "Close." If it's difficult to reuse the form, then a message box showing all the setting values will be sufficient.

A separate issue, can you set the default values of lower and upper elevation to the min and max of DEM during the tool execution mode? Please do so for the PRISM precipitation settings as well.

[lbross]

I'm not sure what you mean by "during the tool execution mode". Do you mean when the form loads (empty) or the clear button is used? Do you want those options to be checked by default? If not checked by default, I could set the default values when the option is checked.

The PRISM min/max doesn't load until the user clicks the 'Get Values' button after choosing the PRISM time period. This is consistent with the Maps dialog.

[jdduh]

I mean when the form is opened from the Auto Pseudo Site button on the Site Scenario Analysis form (instead of the yet-to-be-implemented read-only mode opened from the site list). You can leave the options unchecked. Please also make the circled textboxes wider.

[jdduh]

NWCC requests to limit the search area within AOI. Please use aoi_v as the search boundary.

[lbross]

Made following changes that will be in next build:

1. Set default values for elevation and prism constraints
2. Made textboxes larger as requested
3. Using aoi_v as search boundary

Auto-site logging has not been implemented yet.

[jdduh]

I wasn't able to test the tool because the following issues.

1. The Find Site button is disabled when the Auto-Pseudo site tool first appears. I was able to get pass by enable and disable one of the constraint checkboxes. However, I got this error message after clicking on the Find Site button.

2. If I enable one of constraints, then I got this error message:

[lbross]

The design spec states that if an auto-site was previously created, we load it into the form, disable the find site button, and enable the maps button. This allows the user to see the specs for the previous site and view the map.

If the user changes any field on the form (like enabling/disabling a constraint), we assume they want to create a new auto-site so we enable the find site button and disable the maps button. If we want this to work differently or need some clearer messaging, let me know.

The best way to start a new site is to use the 'Clear' button which resets all of the fields.

I discovered yesterday that the new algorithm did not work if there were no constraint layers set. I believe that is the problem you ran into with the argumentOutOfRange exception. This is fixed in the version I'm currently working on.

I'm not sure what happened with item #2 as elevation constraint has been working for me. Can you take a screenshot of the auto-site form before you submit it so I can see exactly what you are sending that causes the error? Thank you.

[jdduh]

Per item #2, here are the screenshots.

The initial dialog window.

The settings for finding an auto-site.

And the error message after clicking on the find site button.

The error occurs when I only enabled the precipitation constraint. I got an identical error message.

[lbross]

I will look into how the form loads if there is not a previous auto-site. Can you try the elevation again with a value just slightly under the maximum? I wonder if there is some sort of rounding issue where the actual max is less than the value we display and the reclass fails? In my sample AOI, when I use the max value it does work.

[jdduh]

Got a different error message when the elevation range is set to between 2800 and 3700. The AOI I used is Yampa_AOI.

[lbross]

Can you post the AOI to basins? I remember that we had trouble with Yampa before due to some issues with the data. I have a yampa_poly AOI but I'm getting some strange errors about the elevation of the snotel sites when I try to generate maps.

[jdduh]

The AOI is posted at: ftp://basins.geog.pdx.edu/BAGIS/BAGIS_aois/yampa_AOI.zip

[lbross]

I downloaded yampa and was able to run the elevation rule without a hitch. Not sure what you were experiencing yesterday. I posted a new compilation of the add-in (no code changes) just in case something was corrupted. Be sure to quit and restart ArcMap before installing the new version. Here is the map for proof :-)

I also checked into the 'Find Site' button being disabled. If you selected the previous auto-site with no constraints, the form would look like what you are seeing. If you click the maps button, it would show the map with only the represented area. This is confusing. Not sure how we can make it clearer.

[jdduh]

Still get the same error message. I have reset ArcMap (by removing the Normal.mdt file). I did notice that the file size of the newly complied version is 1kb larger than the previous file. I'm using ArcGIS 10.4.1

[lbross]

Are you getting the error 999999 or the earlier 000622? This is the reclass string that I am sending to the Geoprocessor: {2043.51 2800 NoData;2800 3762.37 1;3762.37 3762.37 NoData } I don't know what else to try if we are dealing with some sort of a version compatibility issue. Can you try the PRISM rule and see if it works? It uses much the same framework.

[lbross]

I notice you said the Precipitation had the same issue. Can you try the Discrete Reclass rule in BAGIS-H and see if it works? These use the same method.

[jdduh]

This is the error message from both DEM and PRISM.

BAGIS-H discrete and continuous rules both work normally. My BAGIS-H is based on L037. The current V3 is based on L038.

I will run the tool on ArcMap 10.2 and see if ArcGIS version has an impact.

[jdduh]

In ArcMap 10.2.2, got this error message on DEM, but the PRISM rule works fine without an error.

[lbross]

Maybe try deleting ps_elev, ps_precip in analysis.gdb? These are the targets for the elevation and precip reclass output. They may be in a weird state with the failures. Do you think it is a change with how reclass handles NoData in 10.4? I can try doing a build where we set to -1 instead.

[lbross]

v10.3 is installed on basins. I do not get the error with v10.3. However, I did notice that there was a trailing space on the end of the reclass string which I have removed. I posted a new release to see if this fixes the problem.

I found that v10.3 has a greater degree of precision for the upper/lower bounds of the reclass string. I had been using BA_GetDemStatsGDB(AOIFolderBase) to get the min/max values but apparently this leaves a few pixels out because in 10.3 I am getting some unclassed pixels on the upper/lower ends. I will work on this next.

[jdduh]

Clean up the analysis.gdb did fix the 999999 error on 10.2.2 but not on 10.4.1. I was able to generate a new pseudo site for an elevation range of 2500 to 3500 meters with the new addin. However, if I use the default upper elevation value (3762.37), then I still get the 000622 error.

[jdduh]

Another different behavior with the new addin: after I clicked on the OK button to dismiss the 000622 error message, the 2nd error message box (see an example below) didn't appear and a pseudo-site was created.

[lbross]

Just posted another add-in. I updated the min/max DEM calculations to derive directly from filled_dem rather than use the BA_GetDemStatsGDB(AOIFolderBase). This fixed the problem of unclassed pixels that I was seeing under 10.3. Perhaps it will fix using the default upper elevation 10.4 value as well?

I also added error handling to BA_AddUserFieldToRaster so it won't crash ArcMap anymore. This is failing because the reclass is failing.

Curious as to whether precipitation now works on 10.4. It had the extra space issue as well.

I think the 999999 error is a red herring and we're seeing that after we have a geoprocessor failure.

[jdduh]

The tool now works as spec'd. However, I got this error message when not using any constraints in finding the pseudo site.

See the end of this message for details on invoking just-in-time (JIT) debugging instead of this dialog box.

** Exception Text ** System.ArgumentOutOfRangeException: StartIndex cannot be less than zero. Parameter name: startIndex at System.Text.StringBuilder.Remove(Int32 startIndex, Int32 length) at BAGIS.FrmPsuedoSite.BtnFindSite_Click(Object sender, EventArgs e) in C:\workspace\vb-bagis\src\Maps_Tables\FrmPsuedoSite.vb:line 354 at System.Windows.Forms.Control.OnClick(EventArgs e) at System.Windows.Forms.Button.OnClick(EventArgs e) at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent) at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks) at System.Windows.Forms.Control.WndProc(Message& m) at System.Windows.Forms.ButtonBase.WndProc(Message& m) at System.Windows.Forms.Button.WndProc(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m) at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

[lbross]

This bug is fixed in the BAGIS v3.0-b7. I am glad that the reclass is working on ArcGIS 10.4. I implemented similar logic for the prism reclass just in case.

[jdduh]

NWCC requests a change in the auto-site search domain: perform the analysis using the aoib_v (buffered AOI boundary) but limit the optimal site within the AOI (i.e., aoi_v). That is, when calculating Euclidean distance on the represented areas, the auto-site tool should consider all represented areas even if they are outside AOI.

[lbross]

One way to accomplish this is to mask the Euclidean distance raster to the aoi_v before querying for the cell with the largest distance value. Does this sound like it would return the desired result?