search

geomarker-io / codec

Community Data Explorer for Cincinnati
https://geomarker.io/codec/
GNU General Public License v3.0
4 stars 3 forks source link

codec commons #94

Closed cole-brokamp closed 3 months ago

cole-brokamp commented 4 months ago

Online repository of CoDEC data created by CoDEC developers used for API-based access to data from R or curl.

## login using profile sso account
system2("aws", c("sso", "login", "--profile", "geomarker-io"))
Sys.setenv("AWS_PROFILE" = "geomarker-io")

codec_board() <- function(){
  pins::board_s3(
    bucket = "io.geomarker.codec",
    versioned = FALSE,
    prefix = "data/",
    profile = "geomarker-io",
    cache = tools::R_user_dir("io.geomarker.codec.data", "cache")
  )
}

codec_write() <- function(x) {
  if (!inherits(x, "lbl_tbl")) rlang::abort("x must be a `lbl_tbl` object")
  pins::pin_write(codec_board(), 
                  out, 
                  name = glue::glue("{attr(out, 'name')}_v{attr(out, 'version')}"),
                  type = "rds")
}

codec_read() <- function(x) {
  ...
}
cole-brokamp commented 3 months ago

will use dpkg functions to read and write data packages from s3://geomarker-io/codec_data

host publicly all data derived from code in the CoDEC package; do not "pin" individual-level data

cole-brokamp commented 3 months ago

will write functions to leverage stow_codec_dpkg()

eventually can load all parquet files in S3 CoDEC bucket using duckdb or an AWS tool