Cloud Build triggered from pub/sub

[MarkEdmondson1234]

Do we have examples and support for this? A pub/sub trigger for when a docker container is built and/or Cloud Storage files are ready is a good utility.

https://cloud.google.com/build/docs/automating-builds/create-pubsub-triggers

[MarkEdmondson1234]

This use case covers a Cloud Run version, but Cloud Build would be better for longer batch jobs https://code.markedmondson.me/googleCloudRunner/articles/usecase-r-event-driven-pubsub.html

[MarkEdmondson1234]

Cloud Build triggering from a Cloud Scheduler job would be better than using the current HTTP API call functionality.

Cloud Scheduler -> Pub/Sub topic -> Pub/Sub subscription -> Cloud Build

Could use dependency on googlePubsubR to auto-create the topics/subs https://github.com/andodet/googlePubsubR

[MarkEdmondson1234]

As well as scheduling, can trigger via normal pubsub and use data in the message to add variables to the build.

# create build object
cloudbuild <- system.file("cloudbuild/cloudbuild_substitutions.yml",
                           package = "googleCloudRunner")
the_build <- cr_build_make(cloudbuild)

# create pub/sub topic if needed
library(googlePubsubR)
pubsub_auth()
topics_create("test-topic")

# create build trigger that will work from pub/subscription
pubsub_trigger <- cr_buildtrigger_pubsub("test-topic")
cr_buildtrigger(the_build, name = "pubsub-triggered-subs", trigger = pubsub_trigger)

 # make base64 encoded json
library(jsonlite)                     
message <- list(var1 = "hello mum")
send_me <- base64_enc(toJSON(message))
topics_publish(PubsubMessage(send_me), "test-topic")

# did it work?
cr_build_logs_last("pubsub-triggered-subs")

Yep!

[51] "Step #1 - \"Hello R\": 111255bd9d45: Pull complete"                                                              
[52] "Step #1 - \"Hello R\": 06ab1c1acd68: Pull complete"                                                              
[53] "Step #1 - \"Hello R\": 943eb4dce0a5: Pull complete"                                                              
[54] "Step #1 - \"Hello R\": Digest: sha256:9cb64b052b519bdeee81f1bda9606dad826c5d4dd135a3acadf3fad23b05e5fe"          
[55] "Step #1 - \"Hello R\": Status: Downloaded newer image for rocker/r-base:latest"                                  
[56] "Step #1 - \"Hello R\": docker.io/rocker/r-base:latest"                                                           
[57] "Step #1 - \"Hello R\": [1] \"From PubSub message field1:[\\\"hello mum\\\"]\""                                   
[58] "Finished Step #1 - \"Hello R\""                                                                                  
[59] "PUSH"                                                                                                            
[60] "DONE"                                                                                                            
[61] ""   

[MarkEdmondson1234]

Test set up

cloudbuild <- system.file("cloudbuild/cloudbuild_substitutions.yml",
                          package = "googleCloudRunner")
the_build <- cr_build_make(cloudbuild)

# var1 is sent via Pubsub to the buildtrigger
message <- list(var1 = "hello mum")
send_me <- googlePubsubR::msg_encode(jsonlite::toJSON(message))

# create build trigger that will work from pub/subscription
pubsub_trigger <- cr_buildtrigger_pubsub("test-topic", projectId = "learning-ga4")

cr_buildtrigger(the_build, name = "pubsub-triggered-subs-param", trigger = pubsub_trigger, projectId = "learning-ga4", overwrite = TRUE)

# create scheduler that calls the pub/sub topic with a parameter
cr_schedule("cloud-build-pubsub-params",
            "15 5 * * *",
            pubsubTarget = cr_schedule_pubsub("test-topic",
                                              data = send_me, projectId = "learning-ga4"), projectId = "learning-ga4", overwrite = TRUE)