search

galengorski / ds-pipelines-targets-2

https://lab.github.com/USGS-R/usgs-targets-tips-and-tricks
0 stars 0 forks source link

refactor pipeline to make more intuitive #6

Closed galengorski closed 3 years ago

galengorski commented 3 years ago

I tried to improve this pipeline by creating a target for each individual site. I also broke the site_data target into 4 separate targets: 1) create_directories creates directories where each site's data will be downloaded, 2) site_data_n downloads each site's data and writes it to the directory (one target for each site), 3) site_concat_data reads the downloaded data and concatenates the data into a single data frame, finally 4) site_info_csv queries the site info for the site and writes it to a csv.

galengorski commented 3 years ago

Hi Scott, thanks for the pointers and edits. I have edited the pipeline to address the requested changes. As an aside, I'm not sure the best ways to highlight how the changes that I've made address your edits. Should I do it here? Should I do it as a response to each of your individual comments? Or should I just trust that you can track them down by looking at the changes in the code? Maybe it varies by preference, just curious. Thanks!

github-learning-lab[bot] commented 3 years ago

When you are done poking around, check out the next issue.