search

r-barnes / richdem

High-performance Terrain and Hydrology Analysis
GNU General Public License v3.0
272 stars 70 forks source link
big-data digital-elevation-model geosciences geospatial hydrologic-modeling hydrology

readme

RichDEM

ReadTheDocs Travis DOI

Author: Richard Barnes (rbarnes@umn.edu)

RichDEM is a set of digital elevation model (DEM) hydrologic analysis tools. RichDEM uses parallel processing and state of the art algorithms to quickly process even very large DEMs.

RichDEM offers a variety of flow metrics, such as D8 and D∞. It can flood or breach depressions. It can calculate flow accumulation, slops, curvatures, &c.

RichDEM is available as a performant C++ library, a low-dependency Python package, and a set of command-line tools.

Please cite RichDEM (see below).

Using It

Citing It

As of 883ea734e957, David A. Wheeler's SLOCCount estimates the value of RichDEM at $240,481 and 1.78 person-years of development effort. This value is yours to use, but citations are encouraged as they provide justification of continued development.

General usage of the library can be cited as:

Barnes, Richard. 2016. RichDEM: Terrain Analysis Software. http://github.com/r-barnes/richdem

An example BibTeX entry is:

@manual{RichDEM,
  title        = {RichDEM: Terrain Analysis Software},
  author       = {Richard Barnes},
  year         = {2016},
  url          = {http://github.com/r-barnes/richdem},
}

This information will be updated as versioned releases become available.

Although I have written all of the code in this library, some of the algorithms were discovered or invented by others, and they deserve credit for their good work. Citations to particular algorithms will be printed whenever an app, program, or library function is run. Such citations are prefixed by the character C and look like:

C Barnes, R., Lehman, C., Mulla, D., 2014. Priority-flood: An optimal depression-filling and watershed-labeling algorithm for digital elevation models. Computers & Geosciences 62, 117–127. doi:10.1016/j.cageo.2013.04.024

A typical academic citation might read as follows:

We performed hydrological corrections on our DEM using the Zhou (2016) algorithm implemented in RichDEM (Barnes 2016).

Compilation

To compile, first acquire the source code:

git clone --recursive https://github.com/r-barnes/richdem

You will need a C++ compiler, cmake, and, optionally, libgdal. If you are using conda, you can install these requirements in a new environment with the following commands:

$ conda create -n richdem
$ conda activate richdem
$ conda install --file=requirements.txt -c conda-forge

Then compile using the standard cmake sequence:

$ mkdir build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo ..
$ make -j 6    # Adjust to use more or fewer processors

If you do not want to build richdem with gdal, use the -DUSE_GDAL=OFF option. To install richdem:

$ cmake --install . --prefix /my/install/prefix

If you are using conda, /my/install/prefix should be $CONDA_PREFIX.

RichDEM includes some programs to process extremely large digital elevation models, as well as for handling many DEM formats. Special prerequisites, automatically detected by cmake, are needed for these to compile: GDAL, MPI, and Boost. To install these on a Debian machine use:

sudo apt install openmpi-bin libgdal-dev libopenmpi-dev libboost-iostreams-dev

As A Python Package

From source

After installing the richdem library (as described above), you can build richdem as a Python package. To do this, you will of course need Python (3.7+), which you can install with conda,

$ conda install python

You can now build the richdem Python package,

$ cd wrappers/pyrichdem
$ pip install pybind11
$ python3 setup.py install --user

From conda

To install a pre-built version of richdem using conda,

$ conda install richdem -c conda-forge

From Google Colabs

You can compile and run RichDEM from source in Google Colabs by placing these commands into a cell and running it.

!pip install pybind11
!git clone --recursive https://github.com/r-barnes/richdem.git ghrichdem
!cd ghrichdem/wrappers/pyrichdem && pip install .

From PyPI using pip

Get the package with:

$ pip install richdem

And use:

>>> import richdem

The command:

>>> help(richdem)

provides all the relevant documentation.

As A Command-line Tool

To get the command-line tools, install the Python package with:

pip3 install richdem

The command-line tools are all named rd_*, so typing rd_ on your command- line and hitting tab a few times should give you the full list of what's available.

As A Library

Upon compilation, point your library search path to the include directory. Include various files using, e.g.

#include "richdem/common/Array2D.hpp"

All files include extensive documentation. At this stage the location of certain functions may be subject to change. This will be noted in the NEWS file. (TODO)

As A Handy Collection of Tools

Running make in the apps directory will produce a large number of useful scripts which are essentially wrappers around standard uses of the RichDEM libraries. The apps/README.md file and the apps themselves contain documentation explaining what they all do.

For Processing Large Datasets

The programs directory contains several programs which have not been converted to libraries. This is usually because their functionality is specific and they are unlikely to be useful as a library. Each directory contains a makefile and a readme explaining the purpose of the program.

Documentation

Documentation is available at richdem.com. The documentation is auto-generated from the many README.md files throughout the codebase and the extensive comments in the source code.

Design Philosophy

The design of RichDEM is guided by these principles:

Testing Methodology

Simple algorithms are shown to be correct through visual inspection and comparison against hand-crafted examples. Correctness for more complex algorithms is often "boot-strapped" by comparing the results of simple algorithms versus the complex algorithms on a large number of randomly-generated datasets.

This is a work in progress. TODO

Correctness

Correctness is established via a number of methodologies building from code inspection in the simplest cases to output comparison between simple and complex implementations.

Correctness is noted in source code comments under @correctness sections. These are, in turn, printed to the Doxygen documentation output.

A master list of how correctness was established for each algorithm is available at tests/README.md.

Parsable Output

Every line of output from RichDEM begins with one of the following characters, making it easy to parse with a machine.

All output data shall have the form:

<INDICATOR CHARACTER> <MESSAGE/MEASUREMENT NAME> [= <VALUE> [UNIT]]

The amount of whitespace may very for aesthetic purposes.

Specific Algorithms

Many of the algorithms used in RichDEM are documented in journal or conference publications. In the case of older algorithms by other authors, it is often possible to find the paper in the literature. Some of the newer algorithms I developed have not yet had a chance to be widely utilized. These algorithms are listed below.

Additionally, each publication has its own GitHub repository featuring easily-compiled, minimum working examples of the algorithms, along with examples and test data sets.

These are available as follows:

Publications

The algorithms used in RichDEM have been published in the following articles. Every algorithm/program will provide its relevant citation information when run.

Credits

RichDEM has been developed and tested using computational resources provided by the Minnesota Supercomputing Institute (MSI) and the U.S. National Science Foundation's XSEDE.

Funding for the development of RichDEM has been provided by the Legislative-Citizen Commission on Minnesota Resources (LCCMR), the U.S. National Science Foundation Graduate Research Fellowship, and the U.S. Department of Energy Computational Science Graduate Fellowship.

Feedback

If you see something, say something.

Users are encouraged to report any issues experienced with the code via Github's issue tracker. Feedback is also accepted via email (rbarnes@umn.edu), though this is highly discouraged as it does not provide a resource for others.