FR: CSR (Cloud Source Repositories) EOL June 2024 - move to SSM (Secure Source Manager) as default CICD repository

fmichaelobrien commented 1 month ago

TL;DR

CSR is undergoing deprecation in favour of SSM

Impact to CICD

will affect the CB/CSR default option in the TEF LZ - primarily 0-bootstrap starting with the following gcloud clone
https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/0-bootstrap/scripts/push-to-repo.sh#L32
CB/Cloud Build will continue to be used for the pipeline
will not affect incoming ADO option in #1205
shadow https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/439
reference SSH authentication option over gcloud API clone in https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/431

Cloud Source Repositories is scheduled for end of sale on June 17, 2024. Starting June 17, 2024, if your organization hasn't previously used Cloud Source Repositories, you cannot enable the API or use Cloud Source Repositories. New projects not connected to an organization can’t enable the Cloud Source Repositories API after June 17, 2024. Customers who have already enabled the API prior to this date will not be affected and can continue to use Cloud Source Repositories.

Terraform Resources

1.3.10

Detailed design

Work is in progress in also bringing in ADO (Azure DevOps) as a CI/CD option - as it is the default repository/pipeline tool for 80% of CA PubSec clients

https://github.com/terraform-google-modules/terraform-example-foundation/issues/1205

Additional information

fmichaelobrien will look into the SSM addition unless this work is already assigned in the roadmap

obriensystems commented 1 month ago

module references

https://github.com/terraform-google-modules/terraform-google-bootstrap/blob/master/modules/tf_cloudbuild_source/main.tf

eeaton commented 1 month ago

Thanks for raising this issue. This is identified on our internal roadmap as work to address in the next major round of updates for v5 (sometime in H2 this year). The answer may or may not be SSM, due to the limitation that SSM is currently an invitation only service.

sleighton2022 commented 1 month ago

Closing this issue, and as eeaton noted, we will address this as part of our H2 roadmap.

sleighton2022 commented 1 month ago

Reopening, as I didn't notice it was marked as backlog.

fmichaelobrien commented 1 month ago

Sounds good. I am currently working a PR patch in our fork for later submission.

eeaton commented 1 week ago

One additional aspect to address when we work on this:

It was identified in #1273 that running the docker tests documented in CONTRIBUTING.md also have a dependency on Cloud Build & CSR, so this will fail in any new organizations that haven't previously used CSR. It's not yet clear to me whether we can unpick this locally, or it's an upstream issue with the CFT test framework.

Update: comment from apeabody suggests that this might be the API enablement in the bootstrap project, not necessarily the framework:

Hi @eeaton - I suspect (without seeing diagnostic output) that the make docker_test_prepare dependency on Cloud Source Repositories is due to the sourcerepo API activation in this repo's test/setup: https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/test/setup/main.tf#L65C6-L65C16 That line could likely be commented out to verify or if not using Cloud Source Repositories.

terraform-google-modules / terraform-example-foundation