MarkEdmondson1234 / googleCloudRunner

Easy R scripts on Google Cloud Platform via Cloud Run, Cloud Build and Cloud Scheduler
https://code.markedmondson.me/googleCloudRunner/
Other
81 stars 26 forks source link

Reference secrets from Cloud Run directly #143

Open MarkEdmondson1234 opened 2 years ago

MarkEdmondson1234 commented 2 years ago

Helpful for env variables, auth files etc, will mean less buildsteps for deploying Cloud Run workflows

https://cloud.google.com/run/docs/configuring/secrets

bob-rietveld commented 2 years ago

Hi Mark,

Thanks for making Google Cloud available for R, really great. I had a question about the Secrets. I have a Plumber function where I want to download a file from a gcs bucket, and interface with some other api's (like github).

What is the recommended way to do the authentication for gcs buckets? Below is some example code. Would I need to add a gcs_auth() and reference a secret?

I use cr_deploy_plumber() to deploy the code. ... If I try I see in the logs a error message with

No .httr-oauth file exists in current working directory. Do library authentication steps to provide credentials.

Thanks for your help.

library(googleCloudStorageR)

run <- function( file_name =NULL){

  # stop if no filename is provided
  if(is.null(file_name)){
    stop(
      "No file provided",
      call. = FALSE
    )
  }

  # set bucket
  googleCloudStorageR::gcs_global_bucket("some_bucket_name")

  # read local
  file <- googleCloudStorageR::gcs_get_object( file_name, 
                                                     overwrite = FALSE,
                                                     saveToDisk = file_name)

  #do something with the file

# return name for now
      return(file_name)
}

#' Receive pub/sub message
#' @post /pubsub
#' @param message a pub/sub message
pub <- function(message){

# set global bucket
  file_name <- run(message)

  paste("Echo:", file_name)

}
MarkEdmondson1234 commented 2 years ago

For buckets I suggest making sure the Cloud Run service account has auth access to the bucket as well, then auth via googleAuthR/gargle gar_gce_auth() to reuse the auth running in the environment. This should mean you can avoid uploading service keys etc.

bob-rietveld commented 2 years ago

Thanks for the promt response.

Should I pass in the cloudrunner service account? e.g. googlecloudrunner@project_id.iam.gserviceaccount.com and does it matter if the gar_gce_auth() call live inside or outside of the run function?