No `getters` file? - Githubissues

DOI-USGS / national-flow-observations

This repository pulls national flow data from NWIS

Other

4 stars 8 forks source link

No `getters` file? #22

Closed padilla410 closed 2 years ago

padilla410 commented 2 years ago

Is it a problem that there is no getters file for accessing the shared S3 cache?

> nwis_dv_inventory <- readRDS(sc_retrieve('10_nwis_pull/inout/nwis_dv_inventory.rds.ind'))
Error: while reading 'getters.yml'
The file 'getters.yml' does not exist
In addition: Warning message:
In sc_retrieve("10_nwis_pull/inout/nwis_dv_inventory.rds.ind") :
  sc_retrieve now looks for recipes in 'getters.yml' (the 'scipiper.getters_file' option) by default, but that file does not exist

jordansread commented 2 years ago

It is here, but the default value for the remake_file in that function was set to getters.yml in a past scipiper update. This pipeline probably predates that change.

To get around it, you need to use

nwis_dv_inventory <- readRDS(sc_retrieve('10_nwis_pull/inout/nwis_dv_inventory.rds.ind', remake_file = '10_nwis_pull.yml'))

limnoliver commented 2 years ago

I think you're using the latest version of scipiper, which defaults to remake_file = 'getters.yml'. This repo pre-dates this change, so there are a couple solutions.

We could make a getters.yml file and pull the appropriate targets into that file.
We could revert to an earlier version of scipiper
We could modify those sc_retrieve functions like Jordan indicated above. I think this is the easiest/best option for now.

jordansread commented 2 years ago

As a connection to the other repo/project, the coop pipeline files use a different getter than the default in the lake-temperature-model-prep pipeline. See here

padilla410 commented 2 years ago

Ah great, thanks for the context. I am mostly interested in it because I am developing a little "proof of concept" snippet for my PR. I'll close this.