Update Landlab Component v2 Plan

[ChristinaB]

• [x] Line 23 Update https://github.com/RondaStrauch/pub_strauch_etal_esurf/blob/master/LandslideComponentUsersManual.pdf to live here: https://github.com/landlab/landlab/tree/master/docs/source/user_guide and following this guide. https://github.com/landlab/landlab/blob/master/docs/source/user_guide/overland_flow_user_guide.rst

[ChristinaB]

It looks like your fork is out-of-date with upstream/master and needs updating.

There are syntax errors in landslide_probability.py that are preventing the tests from running.

Add docstrings that demonstrate how to use the new functionality.

Add unit tests that test the new functionality that you've added.

[ChristinaB]

• [x] Ronda - test https://github.com/ChristinaB/pub_strauch_etal_esurf updated synthetic recharge forcing notebook

• [x] Christina - build notebook and test depth to groundwater in the Landlab binder - fork of tutorials https://github.com/landlab/landlab/tree/master/notebooks/tutorials

[ChristinaB]

• [x] update unit test to describe that default is uniform distribution of recharge

• [x] add DTW .py version of landslide component to Binder software env to test and update syntax errors

• [x] Add an example with DTW to the new component

• [x] check DTW uniform and add the working version as the unit test in the ,py

• [x] add recharge and DTW synthetic versions to landlab/notebooks

• [x] finish pull request

[ChristinaB]

• [x] Update final edits to recharge Notebooks in https://github.com/ChristinaB/pub_strauch_etal_esurf

• [x] Create Binder for https://www.hydroshare.org/resource/4cac25933f6448409cab97b293129b4f/ to test Depth to water table in latest Landlab environment. This was updated to PUBLIC so it would run. Added extended metadata to add MyBinder to Open With drop down for anyone who see the resource.

[ChristinaB]

Notes from talking to Eric:

• [x] Make unit test for each version of input recharge x depth x 5 options = 10

• [x] add a calculate known output, at least within a boundary, or not NaN

• [x] avoid deepcopy; instead make a function for creating all the inputs, and run it twice

[ChristinaB]

Earth Day Version

• [x] remove print statements

• [x] extract bounds and plot realistic examples

• [x] update visualizatino to limit plots to [0, 1]

[ChristinaB]

Notes from Landlab call - to reoptimize

• [x] Finalize Code with Synthetic forcings

• TBD what to do first, last or never a) remove mean_watertable_depth output (original 3 outputs) b) add mean_recharge output (increase to 5 outputs) c) split depth and recharge into separate class functions, (4 outputs each) d) review code notes in Word an agree on final outputs

• [x] Add landslide_plot.py

• TBD - to include plotting into the Landlab component or keep with a paper a) plot in a driver/notebook b) .py function with the component c) .py function in landlab/plot (christina's preference)

[ChristinaB]

Logical sequence of next steps; smallest increment documentable

• [ ] 1. incorporate Word notes in existing function in the test notebook. Debug. Test plot run. Test assertions work. Added NaN for recharge output of mean_watertable.

• [x] 2. Add mean_Recharge as an output ( total outputs is 5 - ignore 0 issue)

• [x] 3. add recharge to the plotting

• [x] 4. remove plots from driver and import as a function

• [x] 5. Split the class into 2 (depth and recharge), compare results of plotting, should be identical. Pro: default can be set to (grid) with no other inputs. Con: component may diverge or not be updated in equivalent steps and harder to compare.

OR

• [x] 5. Set missing output to NaN instead of 0 in initialization. Pro: less changes needed right now. May be easier to compare. Con: the code is too long with switches and options.

6. Build unit tests for 1) statistical forcing recharge, 2) data forcing recharge, 3) statistical forcing depth, 4) data forcing depth.

[ChristinaB]

Things to test:

• [x] Missing Ksat; what happens?

• [x] Missing Cmin and Cmax; what happens?

• [x] Test Is Not None recharge_lognormal_mean - what happens if the mean isn't given?

• [x] What happens if input array is a different size that node length for lognormal spatial and Data driven spatial recharge/depth ? Does it need to be added for data driven spatial?

[ChristinaB]

Testing assertions for missing inputs:

[ChristinaB]

Discussed new sets of parameters with Erkan to parameterize De and Re based on known range of Relative Wetness 0-1

[ChristinaB]

The code updates and unit tests are here for component.py and here for the run unit test notebook, debugged for true/false options, added saturation threshold definition of 0.001 meters depth to groundwater is "saturated" for the use in RW calculations. Fixed limits on plotting. The last scenario in the unit tests is the one that sets the RW based min, max, mean, std that are used throughout the rest of the d, r, and statistical plots.

[ChristinaB]

• [x] Add markdown to unit test sanity test readable be learning humans

• [x] build .py for actual unit test

[ChristinaB]

• [x] recharge in unit test is m/d and the component is expecting mm/d (unit issue?)
• [x] Does De_stdve = soilthickness - hw.stdev; seems like it will be too high?
• [ ] Still have a problem if we don't provide ksat as a grid input. We should be able to get this to work with just transmissivity.
• [x] Are these the plots we want to come out for the unit test...no variability? and no failure?

Build functions to create grids to for unit test functions that solve for single and variable inputs and areas - - this is to control for and solve a given set of inputs for Relative Wetness =1 for any given hydrology forcing. This is independent of the factor of safety unit test. We are doing this so we dont go crazy analyzing 7 million grid cells and proving we did it correctly.

Vision: input a real grid to create a synthetic grid that captures the statistics and range of the real area, and analyze the range of inputs outputs and parameters for a model instance unit test.

[ChristinaB]

• [ ] Box 1 – should we say that this function is using defaults..those are what we have in last line.

• [ ] Box 2 - ‘Asign’ spelling; should we say the grid has nodes spaced 10 m apart? Landlab nodes vs cells

• [ ] Box 5 – maybe delete first print and add, For this customized bigger grid,” remove ‘only’; put the run time and computational cost as questions to pose in the mark up above.

• [ ] Box 6 - What is the purpose of this box...no markup comments. Might use the non unit grid

• [ ] Box 8 – why define ‘dimensions’ 2X? Need in box 5?

• [ ] Box 10 – what is ‘getters’

• [ ] Box 11 – shouldn't there be some output, return?

• [ ] Box 13 – same thing as 11

• [ ] Switch box 16 & 17

• [ ] Box 18 – return something?

• [ ] Box 20 – what's going on here?, see comment line above

• [ ] Box 23 – remove extra space

• [ ] Box 25 – no markdown comment...what is this doing?

• [ ] Box 26 – same thing, what is LaTex about?

• [ ] Section 1.2 - nothing after it?

• [ ] Box 28/29 - not 100x100 grid; does T look right?; why ‘max()’

• [ ] Box 30 – needs markdown...introduce the concept of wet and dry scenarios; recharge std look large (meters?), relative wetness from recharge of by 3 decimals? What does Scenario_R equal...results of the explorer

• [ ] Box 31 – introduce distributions, why ‘n’ here and later?

• [ ] Box 32 – explain why before and after arrays differ...performed the calculation

• [ ] Box 36 – unit test calculations – markdown doesn’t match

• [ ] Box 38 – delete

• [ ] Box 40– remind folks where gidnum comes from. If this wasn’t the synthetic unit grid, this might error

• [ ] Box 41 – this option only works for 5,4 grid; see note, is this a much lower recharge than uniform?

• [ ] Box 43 – remove last print

• [ ] Markdown above ‘generate depth to water table...this goes down before box 45

• [ ] What is the mark downs and unit test doing in “Run the landslide component” What are we trying to do here?

• [ ] Box 50/51 - isn’t this repeating uniform?

• [ ] Box 52 – delete

• [ ] Add markdown when moving between uniform, lognormal,…

• [ ] Box 64 – should node 6 be .92?

• [ ] Box 69 – check on naming of recharge model run...unit?

• [ ] Box 81 – subtitle spelling of Probability – for failure; Add print statement that this is for depth;

• [ ] Worried about scenarios – we have wet/dry and depth/recharge

• [ ] Box 84 - think data-driven spatial doesn’t have a /1000 for Re

[ChristinaB]

• [ ] how to cite tutorial

• [ ] !pip freeze

• [ ] twitter

• [ ] SCL funding and logo

[ChristinaB]

5/15/2020 Update

1. Asked CSDMS team for help with PR: https://github.com/landlab/landlab/pull/1180 Pull request updated:
2. Architected how to use HydroShare to launch a Binder environment using a Landlab development space. Unit_test_demo.ipynb can be launched here: this is easier than using Binder when the branch is not yet merged with the distributed master version of landlab.
3. Sorting out instructions for developing access to interactive coding that enables software developers to test code with domain scientists: Updated https://github.com/waterhackweek/learning-resources/blob/master/docs/hydroshare_binder.md

[ChristinaB]

• [x] Component Definitions – switch hydrologic forcings and distribution options in paragraph

• [ ] What is the difference 4&5?

• [ ] Let’s go over numbering – 1., 1.1, T1 Later on we have 1.3, 1.4, 1.5...we don’t actually run these?

• [ ] If the output variables are initialized to 0 (T11), could these values carry over after model run, like could they not be overwritten?

• [ ] Box 26 – mean water table depth 0-1000?

• [ ] Box 95 – T is m2/day, not m/day; T be more for deeper soils (if same Ksat)

• [ ] Box 66 – think 2nd section should be ‘Unit Depth Parameters m:’

• [ ] Box 67 – flip depth and recharge...keep consistency (same thing with T13 & T14...)

• [ ] 2.3 - seems like we need a header here that we are going into the distribution options

• [ ] Box 34 – might folks not know where ‘unit_R[0]’ comes from?

• [ ] Box 39 – Don't think we need the markdown ‘***Note...” anymore

• [ ] HSD dictionaries for R only work for the 5x4 grid as we specify the core number specifically

• [ ] Box 41 – ADD ‘key, such as for 5th core node:’

• [ ] Need a bit more explanation in 2.6.2 for this dictionary compared to previous, and it is a single parameter (vs. 3 dict).

• [ ] Box 86 – what is associated with these core nodes? Is this probability of failure?

• [ ] Box 77 – why is core node 10 = 0.92, should this be 0.96? (starts with 0 node), but 0 node is ‘NA’ in reality, not 0 (see comment #4)

• [ ] 4.4 - the initialized probability, isn’t that 0?

[ChristinaB]

Binder for Landlab - latest version on github/landlab

The end goal is to have these three files (and others) working with this Binder.

Launch the Landlab Binder https://mybinder.org/v2/gh/ChristinaB/landlab/ChristinaB-landslide by clicking on the badge below.

don't click on this now don't use this now The files that will be used in the Landlab PR are only, will only work once these three are finalized and in sync with Landlab: Unit_Test_Landslides.ipynb Synthetic_Landslide_Component_Default.ipynb Synthetic_Landslide_Component_Options.ipynb

Binder for Development Testing - pre Landlab integration

We have to make a new Binder and Edit all the rest of the files and upload to the bindertest-landslide branch. The .py files are in this location only to make the Notebooks run on Binder before the Landlab PR and release. Click here to launch a binder to test and compare in a binder with Landlab installed, but the Notebooks set up to import the changed files from within the Landslide tutorials folder. click on this badge ) click on this badge This takes a while to build.
Navigate to notebooks/tutorials/landslides

[ChristinaB]

How did I do that?

MyBinder.org

Github: https://github.com/ChristinaB/landlab Branch: bindertest-patch-1 Path to Notebook: https://github.com/ChristinaB/landlab/blob/bindertest-patch-1/notebooks/tutorials/landslides/Synthetic_Landslide_Component_Default-Binder4devtest.ipynb

[ChristinaB]

https://meet.jit.si/mybinder.org-2134f8d90c7e0f7a46ff9d8f9c32862b#config.startWithVideoMuted=true&config.startWithAudioMuted=true&config.prejoinPageEnabled=true

[ChristinaB]

‘Default’ notebook – runs just for Uniform distribution

• [x] Box 5, note that you got input var names without running component

• [x] You have commented out code boxes – duplicate box 14

• [x] Box 23 – why print out failure 2x. core node print out is only for depth

• [x] Final plot...should this be soil__mean_recharge_mm? note the 'mm'

‘Option’ notebook – runs 4 distribution options

• [ ] Put the purpose of the notebooks in the beginning, a bit more than the bold statement (both notesbooks)

• [ ] Where do the default values come from, folks won't know. These are not the component default, but ‘test’ defaults??

• [ ] Call lots of new functions, but not all described so it might appear a bit ‘black box’ to fresh eyes

• [ ] We'll have to explain why depth probability of saturation doesn’t appear - in a comment box

Section 4.4 is not 'running the component'

[ChristinaB]

To run NOT on Binder:

1. Add a server to your App Open With visibility. Click red grid to green at https://www.hydroshare.org/resource/6d7ef1127ad04f019fe11714daf47e02/

2. Go to your HS resource: https://www.hydroshare.org/resource/4cac25933f6448409cab97b293129b4f/

3. Open With - hsjp07.cigi.illinois.edu is the server running the landlab paper

1b -- get invite to the compute group to XSEDE - this connects to Open With CyberGIS for Water This is jetstream server connected to XSEDE

1c -- use CUAHSI JH for small jobs - no extra steps required.

1d -- set CSDMS as an App on HydroShare and set up user access with Eric.

[ChristinaB]

What drives a Landlab Landslide? A hydrology model.

How does Landlab hydro a Landslide? Recharge and Depth to Water Table.

Landlab Landslide Component Demonstration with a Synthetic Grid and implementation with Two Hydrologic Forcing Options (Depth to Groundwater and Recharge) compared using default parameters and Uniform Distribution.**

**

[ChristinaB]

To run Slippery Future Notebooks on HydroShare, just like the good ole days:

• [x] 0. Get permission to compute on the CUAHSI Compute Group.

The CUAHSI JupyterHub is a default available on Open With. To see other servers -- for example: Add a server to your App Open With visibility. Click red grid to green at https://www.hydroshare.org/resource/6d7ef1127ad04f019fe11714daf47e02/

1. Go to the Slippery Future HS resource: https://www.hydroshare.org/resource/4cac25933f6448409cab97b293129b4f/

2. Open With - CUAHSI JupyterHub

3. Select a Server: Python 3.7 Scientific with Landlab included in the software environment

4. If default view takes you to Jupyter Lab, just replace lab in the URL to tree. Or watch a tutorial and learn to navigate with tree or lab views of Folders and Notebooks.

[ChristinaB]

Fixed kernel errors! 'pywin32_postinstall.py -install'