Change push location from GD to S3

DOI-USGS / national-flow-observations

This repository pulls national flow data from NWIS

Other

4 stars 8 forks source link

Change push location from GD to S3 #1

Closed limnoliver closed 4 years ago

limnoliver commented 4 years ago

Should push this data to an S3 bucket for broader accessibility and more storage. @lindsayplatt to provide an example pattern to replicate.

lindsayplatt commented 4 years ago

Example with 3 functions - two that push data to s3 and one that pulls from s3. https://github.com/usgs-makerspace/makerspace-sandbox/blob/b776e30c717ef0419ff413591f24bdbd6fe3ed31/WBEEP/rscripts/generate_gif_images/interact_with_s3.R

This is the corresponding scipiper yml that goes with those functions: https://github.com/usgs-makerspace/makerspace-sandbox/blob/b776e30c717ef0419ff413591f24bdbd6fe3ed31/wbeep_model_map_gif.yml

lindsayplatt commented 4 years ago

Created ds-pipeline-national-flow-observations under the dev VPC.

aws.signature::use_credentials(profile='default', file=aws.signature::default_credentials_file())
aws.s3::bucket_exists("ds-pipeline-national-flow-observations")
[1] TRUE