GoogleCloudPlatform / cloud-builders-community

Community-contributed images for Google Cloud Build
https://cloud.google.com/cloud-build/
Apache License 2.0
1.26k stars 858 forks source link

Add a GCS lock builder #520

Open jsok opened 3 years ago

jsok commented 3 years ago

Additional Info

A common problem people encounter when using Cloud Build is the lack of concurrency control between builds. Coordinating builds to ensure they have exclusive access to resources (git tags, deployments, etc.) is a hard problem to solve in general.

One approach is to provide some locking mechanism so that builds can acquire and release a lock before entering a critical section and releasing once completed.

GCS's consistency along with particular request headers can be used to implement a lock. A popular (bit simplistic) implementation is: https://github.com/mco-gh/gcslock A more thorough analysis on the topic can be found at: https://www.joyfulbikeshedding.com/blog/2021-05-19-robust-distributed-locking-algorithm-based-on-google-cloud-storage.html

Example

steps:
- name: gcr.io/cloud-builders-community/gcs-lock-acquire
  args: ['gs://bucket/lock']
- id: deployment-step
  name: ...
  args: ['my', 'deployment', 'arguments']
- name: gcr.io/cloud-builders-community/gcs-lock-release
  args: ['gs://bucket/lock']