MarkEdmondson1234 / googleCloudRunner

Easy R scripts on Google Cloud Platform via Cloud Run, Cloud Build and Cloud Scheduler
https://code.markedmondson.me/googleCloudRunner/
Other
81 stars 26 forks source link

cr_schedule failing on scheduled builds - INVALID_ARGUMENT - issue with cr_build_schedule_http() #98

Open MarkEdmondson1234 opened 3 years ago

MarkEdmondson1234 commented 3 years ago

Examples like this work so its not authentication

cloudbuild <- system.file("cloudbuild/cloudbuild.yaml", package = "googleCloudRunner")
build1 <- cr_build_make(cloudbuild)

cr_schedule("cloud-build-test1", schedule="15 5 * * *", httpTarget = cr_build_schedule_http(build1))

It must be the parsing of the cloud build steps somehow, in cr_build_schedule_http()

This didn't work, but should:

# can be R file location or in-line code
r_code <- c("x <- rnorm(1:10)",
            "dir.create('data-raw')",
            "save(x, file = paste0('data-raw/data_', make.names(Sys.time()), '.Rda'))")

bs <- c(
  # assumes you have git credentials saved to Secret Manager
  cr_buildstep_gitsetup("github-ssh"),
  cr_buildstep_git(c("clone",
                     "git@github.com:$_GITHUB_REPO",
                     "payload")),
  # run the R code above - could also be a file in the github repo
  cr_buildstep_r(r_code, id = "run r code", dir = "payload"),
  # could put the git commands in one cr_buildstep_bash step for brevity
  cr_buildstep_git(c("add","--all"), dir = "payload"),
  cr_buildstep_git(c("commit", "-a", "-m", 
                     "Ran R script to add data $BUILD_ID"),
                   dir = "payload"),
  cr_buildstep_git(c("push", "git@github.com:$_GITHUB_REPO"), dir = "payload")
)

# using a substitution var for the GitHub repo to make it more portable
build_yaml <- cr_build_yaml(
  steps = bs,
  substitutions = list(`_GITHUB_REPO`="MarkEdmondson1234/r-code-commit-github")
)

# builds ok
build <- cr_build(build_yaml)

cr_schedule("r-2-git", schedule="15 5 * * *", httpTarget = cr_build_schedule_http(build))

The schedule gets created ok, but on execution there is an unhelpful error:

@type: "type.googleapis.com/google.cloud.scheduler.logging.AttemptFinished"   
  jobName: "projects/xxx/locations/europe-west1/jobs/r-2-git-demo"   
  status: "INVALID_ARGUMENT"   
  targetType: "HTTP"   
  url: "https://cloudbuild.googleapis.com/v1/projects/xxx/builds"   
MarkEdmondson1234 commented 3 years ago

It works using cr_build_make()

# your running successful build
build <- cr_build_make(build_yaml)

# create build API call
schedule_me <- cr_build_schedule_http(build)

# schedule the build
cr_schedule("r-2-git-demo", "5 15 1 * *", httpTarget = schedule_me)
#==CloudScheduleJob==
#name:  projects/xxxx/locations/europe-west1/jobs/r-2-git-demo 
#state:  ENABLED 
#httpTarget.uri:  https://cloudbuild.googleapis.com/v1/projects/xxxx/builds 
#httpTarget.httpMethod:  POST 
#userUpdateTime:  2021-01-01T07:21:44Z 
#schedule:  5 15 1 * * 
#timezone:  Europe/Copenhagen 

but not the object returned from cr_build() or cr_build_status()