Get started with USGS Data Science pipelines

[github-learning-lab[bot]]

Data analyses are often complex. Data pipelines are ways of managing that complexity. Our data pipelines have two foundational pieces:

• Good organization of code scripts help you quickly find the file you need, whether you or a teammate created it.

• Dependency managers such as remake, scipiper, and drake formalize the relationships among the datasets and functions to ensure reproducibility while also minimizing the amount of unnecessary runtime as you're creating or modifying parts of the pipeline.

:keyboard: Activity: Assign yourself to this issue to get started.

:bulb: Tip: Throughout this course, I, the Learning Lab Bot, will reply and direct you to the next step each time you complete an activity. But sometimes I'm too fast when I :hourglass_flowing_sand: give you a reply, and occasionally you'll need to refresh the current GitHub page to see it. Please be patient, and let my humans know (jread-usgs or aappling-usgs) if I seem to have become completely stuck.

---

I'll sit patiently until you've assigned yourself to this one.

[github-learning-lab[bot]]

:keyboard: Installing tools

To complete this section, you'll need some R packages. You will need to install remake, scipiper, and drake.

drake is available through CRAN, but remake and scipiper both require installation from github.

scipiper depends on remake, so install remake and its dependencies first:

install.packages(c("R6", "yaml", "digest", "crayon", "optparse", "storr", "remotes"))
remotes::install_github('richfitz/remake')

While you are at it, please install a few other packages that you'll use along the way:

install.packages(c("dplyr", "readr", "stringr", "sbtools", "whisker", "dataRetrieval", "ggplot2", "purrr"))

Next, install scipiper

remotes::install_github('USGS-R/scipiper')

Also install drake

install.packages('drake')

When all three pipeline packages are installed, comment with the message that appears after you run library(scipiper).

---

I'll respond when I detect you've commented to this issue.

[sspaulding777]

USGS Support Package: https://owi.usgs.gov/R/packages.html#support

[github-learning-lab[bot]]

Your start-up message looks good, @sspaulding777.

Take a look at the link in the message here if you haven't already. Note that we have various types of R packages within USGS with different levels of user support and difference audiences. The scipiper tool is one that we share publicly but we don't intend to support it widely.

---

When you are done poking around, check out the next issue.