search

GoogleCloudPlatform / pbmm-on-gcp-onboarding

GCP Canadian Public Sector Landing Zone overlay on top of the TEF via CFT modules - a secure cloud foundation
https://cloud.google.com/architecture/security-foundations
Apache License 2.0
39 stars 55 forks source link

Add Azure DevOps ADO as build and repository option #399

Open obriensystems opened 2 months ago

obriensystems commented 2 months ago

Work Items

20240502: note CB/CSR option details in https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/wiki#20240502-repo-state

The default is Cloud Build and Cloud Source Repositories. We will also support alternatives like Github and ADO

Documentation on https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/wiki/DevOps#azure-devops

Branch / Issues

Test accounts

Terminals

Artifacts to migrate from CB to ADO

Create new ADO org

Screenshot 2024-04-23 at 11 25 53

select private project

Screenshot 2024-04-23 at 11 28 22

Usage

https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/tef-gcp-pbmm-lz/_git/tef-gcp-pbmm-lz

git remote add origin https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/tef-gcp-pbmm-lz/_git/tef-gcp-pbmm-lz
git push -u origin --all
fmichaelobrien commented 2 months ago

Procedure to create a service account, key and provision a container based ADO pipeline to do canary operations on the GCP account

On the GCP account

Create service account in bootstrap project (out of band of terraform for now)

michael@cloudshell:~/tef-olxyz/github/pbmm-on-gcp-onboarding (tef-olxyz)$ gcloud iam service-accounts create ado-manual-tef-olxyz-remote-sa --display-name "ADO Manual TEF olxyz remote SA"
Created service account [ado-manual-tef-olxyz-remote-sa].

Test clone/pull/push On a local laptop (M1 mac in this case

generate credentials

Screenshot 2024-04-23 at 16 47 16 
michaelobrien@mbp7 olxyz % git clone https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/tef-gcp-pbmm-lz/_git/tef-gcp-pbmm-lz
Cloning into 'tef-gcp-pbmm-lz'...
Password for 'https://obrienlabsxyz@dev.azure.com': 
warning: You appear to have cloned an empty repository.
michaelobrien@mbp7 olxyz % ls
azure       tef-gcp-pbmm-lz

michaelobrien@mbp7 olxyz % cd tef-gcp-pbmm-lz 
michaelobrien@mbp7 tef-gcp-pbmm-lz % ls
michaelobrien@mbp7 tef-gcp-pbmm-lz % vi README.md
michaelobrien@mbp7 tef-gcp-pbmm-lz % git status
On branch main

No commits yet

Untracked files:
  (use "git add <file>..." to include in what will be committed)
    README.md

nothing added to commit but untracked files present (use "git add" to track)
michaelobrien@mbp7 tef-gcp-pbmm-lz % git add README.md 
michaelobrien@mbp7 tef-gcp-pbmm-lz % git status
On branch main

No commits yet

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)
    new file:   README.md

michaelobrien@mbp7 tef-gcp-pbmm-lz % git commit -m "#1 - test push"
[main (root-commit) a8de8ce] #1 - test push
 1 file changed, 2 insertions(+)
 create mode 100644 README.md
michaelobrien@mbp7 tef-gcp-pbmm-lz % git push origin main
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Writing objects: 100% (3/3), 238 bytes | 238.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Analyzing objects... (3/3) (3 ms)
remote: Validating commits... (1/1) done (0 ms)
remote: Storing packfile... done (100 ms)
remote: Storing index... done (59 ms)
To https://dev.azure.com/obrienlabsxyz/tef-gcp-pbmm-lz/_git/tef-gcp-pbmm-lz
 * [new branch]      main -> main
michaelobrien@mbp7 tef-gcp-pbmm-lz %
Screenshot 2024-04-23 at 16 51 22
obriensystems commented 2 months ago

Procedure: upstream github fork is pushed to ADO

In ADO

create a new project

pbmm-on-gcp-onboarding-fork

Repos | import

Screenshot 2024-04-23 at 17 01 54

use https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git

Screenshot 2024-04-23 at 17 02 32

After import https://dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding-fork/_git/pbmm-on-gcp-onboarding-fork

Screenshot 2024-04-23 at 17 03 45

Generate Git Credentials

Screenshot 2024-04-23 at 17 04 50

clone

michaelobrien@mbp7 azure % git clone https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding-fork/_git/pbmm-on-gcp-onboarding-fork
Cloning into 'pbmm-on-gcp-onboarding-fork'...
remote: Azure Repos
remote: Found 6394 objects to send. (14 ms)
Receiving objects: 100% (6394/6394), 31.89 MiB | 22.52 MiB/s, done.
Resolving deltas: 100% (3886/3886), done.
michaelobrien@mbp7 azure % cd pbmm-on-gcp-onboarding-fork 
ichaelobrien@mbp7 pbmm-on-gcp-onboarding-fork % git status
On branch 243-tef-retrofit
Your branch is up to date with 'origin/243-tef-retrofit'.

nothing to commit, working tree clean
michaelobrien@mbp7 pbmm-on-gcp-onboarding-fork % git checkout main
branch 'main' set up to track 'origin/main'.
Switched to a new branch 'main'
michaelobrien@mbp7 pbmm-on-gcp-onboarding-fork % ls        
0-bootstrap         4-projects          LICENSE             fix_tfvars_symlinks.py      test
1-org               5-app-infra         Makefile            go.work             z_2024_v020_pre_tef_v4
2-environments          CHANGELOG.md            README.md           helpers
3-networks-dual-svpc        CONTRIBUTING.MD         build               policy-library
3-networks-hub-and-spoke    ERRATA.md           docs                scripts

Procedure to pull from upstream origin

michaelobrien@mbp7 pbmm-on-gcp-onboarding-fork % git remote rm  github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git     
error: No such remote: 'github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git'
michaelobrien@mbp7 pbmm-on-gcp-onboarding-fork % git remote rm upstream                                                  
michaelobrien@mbp7 pbmm-on-gcp-onboarding-fork % git remote add upstream https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git
michaelobrien@mbp7 pbmm-on-gcp-onboarding-fork % git fetch upstream
From https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding
 * [new branch]      243-tef-retrofit                                                          -> upstream/243-tef-retrofit
 * [new branch]      318-log-sink-alerting                                                     -> upstream/318-log-sink-alerting
 * [new branch]      332-dev-prov-client-v20230917                                             -> upstream/332-dev-prov-client-v20230917
 * [new branch]      341-labels-off-332                                                        -> upstream/341-labels-off-332
 * [new branch]      341-tags-off-332                                                          -> upstream/341-tags-off-332
 * [new branch]      345-landing-zone-fortigate-light                                          -> upstream/345-landing-zone-fortigate-light
 * [new branch]      351-landing-zone-fortigate-cloud-setup                                    -> upstream/351-landing-zone-fortigate-cloud-setup
 * [new branch]      craigenator                                                               -> upstream/craigenator
 * [new branch]      craigenator-network                                                       -> upstream/craigenator-network
 * [new branch]      dan-fix-version-upgrade                                                   -> upstream/dan-fix-version-upgrade
 * [new branch]      dependabot/go_modules/helpers/foundation-deployer/golang.org/x/net-0.23.0 -> upstream/dependabot/go_modules/helpers/foundation-deployer/golang.org/x/net-0.23.0
 * [new branch]      dependabot/go_modules/helpers/foundation-deployer/google.golang.org/protobuf-1.33.0 -> upstream/dependabot/go_modules/helpers/foundation-deployer/google.golang.org/protobuf-1.33.0
 * [new branch]      dependabot/go_modules/test/integration/golang.org/x/net-0.23.0            -> upstream/dependabot/go_modules/test/integration/golang.org/x/net-0.23.0
 * [new branch]      example                                                                   -> upstream/example
 * [new branch]      fmichaelobrien                                                            -> upstream/fmichaelobrien
 * [new branch]      fmichaelobrien-example-nonprod                                            -> upstream/fmichaelobrien-example-nonprod
 * [new branch]      fmichaelobrien-example-prod                                               -> upstream/fmichaelobrien-example-prod
 * [new branch]      fmichaelobrien-pr-113                                                     -> upstream/fmichaelobrien-pr-113
 * [new branch]      fortigate-yopps                                                           -> upstream/fortigate-yopps
 * [new branch]      gh357-tef-v4-fork                                                         -> upstream/gh357-tef-v4-fork
 * [new branch]      gh360-day0-deploy-example                                                 -> upstream/gh360-day0-deploy-example
 * [new branch]      gh362-bootstrap-prereq                                                    -> upstream/gh362-bootstrap-prereq
 * [new branch]      gh377-architecture                                                        -> upstream/gh377-architecture
 * [new branch]      gh399-ado                                                                 -> upstream/gh399-ado
 * [new branch]      hotfix/cloudbuildjob                                                      -> upstream/hotfix/cloudbuildjob
 * [new branch]      hotfix/guardrailsiam                                                      -> upstream/hotfix/guardrailsiam
 * [new branch]      main                                                                      -> upstream/main
 * [new branch]      pr-mro-bootstrap-localization                                             -> upstream/pr-mro-bootstrap-localization
 * [new branch]      pr-mro-bootstrap-run-terraform-local-041524                               -> upstream/pr-mro-bootstrap-run-terraform-local-041524
 * [new branch]      pr-mro-tef-fix-bucket-name-length-041524                                  -> upstream/pr-mro-tef-fix-bucket-name-length-041524
 * [new branch]      pr-mro-tef-multi-instance-041524                                          -> upstream/pr-mro-tef-multi-instance-041524
 * [new branch]      tm-lz-org                                                                 -> upstream/tm-lz-org
 * [new branch]      v020                                                                      -> upstream/v020
 * [new branch]      v20240407                                                                 -> upstream/v20240407
michaelobrien@mbp7 pbmm-on-gcp-onboarding-fork % git merge upstream/main main
Already up to date.
michaelobrien@mbp7 pbmm-on-gcp-onboarding-fork % git status         
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean

Pull changes from upstream to test - push to remote ado

https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/pull/378

michaelobrien@mbp7 pbmm-on-gcp-onboarding-fork % git fetch upstream                                                                       
remote: Enumerating objects: 7, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 7 (delta 2), reused 1 (delta 1), pack-reused 0
Unpacking objects: 100% (7/7), 5.08 KiB | 1.02 MiB/s, done.
From https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding
   5a5c057..3d9f3e8  gh377-architecture -> upstream/gh377-architecture
   07100b1..b9fe8d3  main               -> upstream/main
michaelobrien@mbp7 pbmm-on-gcp-onboarding-fork % git merge upstream/main main
Updating 07100b1..b9fe8d3
Fast-forward
 docs/architecture.md | 138 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 138 insertions(+)
 create mode 100644 docs/architecture.md
michaelobrien@mbp7 pbmm-on-gcp-onboarding-fork %

Push upstream to main

michaelobrien@mbp7 pbmm-on-gcp-onboarding-fork % git push origin main
Enumerating objects: 12, done.
Counting objects: 100% (12/12), done.
Delta compression using up to 10 threads
Compressing objects: 100% (7/7), done.
Writing objects: 100% (7/7), 4.18 KiB | 4.18 MiB/s, done.
Total 7 (delta 4), reused 0 (delta 0), pack-reused 0
remote: Analyzing objects... (7/7) (5 ms)
remote: Validating commits... (3/3) done (1 ms)
remote: Storing packfile... done (57 ms)
remote: Storing index... done (102 ms)
To https://dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding-fork/_git/pbmm-on-gcp-onboarding-fork
   07100b1..b9fe8d3  main -> main
Screenshot 2024-04-23 at 17 29 32
obriensystems commented 2 months ago

Setup ADO Billing or 30 day free trial

Screenshot 2024-04-23 at 19 47 32

Test example pipeline and GCP access before 0-bootstrap run

Screenshot 2024-04-23 at 19 49 01

get grant

Screenshot 2024-04-23 at 19 49 50

skip https://aka.ms/azpipelines-parallelism-request go straight to paid account

post subscription conversion https://learn.microsoft.com/en-us/answers/questions/477716/how-to-resolve-no-hosted-parallelism-has-been-purc

Screenshot 2024-04-23 at 20 48 24 Screenshot 2024-04-23 at 20 49 02 Screenshot 2024-04-23 at 20 49 49 Screenshot 2024-04-23 at 20 51 15 Screenshot 2024-04-23 at 20 52 40

response to parallel jobs submitted 20240423:2255

self hosted agent as backup

project settings | agent pools | add self-hosted

Screenshot 2024-04-23 at 22 57 58

new agent - osx - download

michaelobrien@mbp7 azure % mkdir ado-agent
michaelobrien@mbp7 azure % cp vsts-agent-osx-x64-3.238.0.tar.gz ado-agent 
michaelobrien@mbp7 ado-agent % tar -xvf vsts-agent-osx-x64-3.238.0.tar.gz 

haelobrien@mbp7 ado-agent % ./config.sh
./config.sh: line 93:  6035 Killed: 9               ./bin/Agent.Listener configure "$@"

switching to windows agent
obriensystems commented 2 months ago

windows local agent

PS C:\opt>  mkdir agent ; cd agent

    Directory: C:\opt

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----        2024-04-23  11:13 PM                agent

PS C:\opt\agent> Add-Type -AssemblyName System.IO.Compression.FileSystem ; [System.IO.Compression.ZipFile]::ExtractToDirectory("$HOME\Downloads\vsts-agent-win-x64-3.238.0.zip", "$PWD")
PS C:\opt\agent> ls

    Directory: C:\opt\agent

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----        2024-04-23  11:13 PM                bin
d-----        2024-04-23  11:13 PM                externals
-a----        2024-04-09   4:26 PM           2967 config.cmd
-a----        2024-04-09   4:26 PM           3190 run.cmd

image

use a PAT from user settings image

image

getting https://learn.microsoft.com/en-us/answers/questions/272411/vs30063-you-are-not-authorized-to-access-https-dev

need trailing slash

regenerated token with all-access - working


>> Connect:

Enter server URL > https://dev.azure.com/obrienlabsxyz/
Enter authentication type (press enter for PAT) >
Enter personal access token > ****************************************************
Connecting to server ...

>> Register Agent:

Enter agent pool (press enter for default) > olxyz-self
Enter agent name (press enter for 13900D) >
Scanning for tool capabilities.
Connecting to the server.
Successfully added the agent
Testing agent connection.
Enter work folder (press enter for _work) >
2024-04-24 03:34:21Z: Settings Saved.
Enter run agent as service? (Y/N) (press enter for N) > y
Enter enable SERVICE_SID_TYPE_UNRESTRICTED for agent service (Y/N) (press enter for N) > y
Enter User account to use for the service (press enter for NT AUTHORITY\NETWORK SERVICE) >
Granting file permissions to 'NT AUTHORITY\NETWORK SERVICE'.
Service vstsagent.obrienlabsxyz.olxyz-self.13900D successfully installed
Service vstsagent.obrienlabsxyz.olxyz-self.13900D successfully set recovery option
Service vstsagent.obrienlabsxyz.olxyz-self.13900D successfully set to delayed auto start
Service vstsagent.obrienlabsxyz.olxyz-self.13900D successfully set SID type
Service vstsagent.obrienlabsxyz.olxyz-self.13900D successfully configured
Enter whether to prevent service starting immediately after configuration is finished? (Y/N) (press enter for N) > y

image

wrong pool image

turn off default agent image

local agent was not running

# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger:
- main

pool: olxyz-self
#  vmImage: ubuntu-latest
#  agent.name: 13900D

steps:
- script: echo Hello, world!
  displayName: 'Run a one-line script'

- script: |
    echo Add other tasks to build, test, and deploy your project.
    echo See https://aka.ms/yaml
  displayName: 'Run a multi-line script'

results
PS C:\opt\agent> .\run.cmd
Scanning for tool capabilities.
Connecting to the server.
2024-04-24 03:43:31Z: Listening for Jobs

image

force pool in yml

pool: olxyz-self
#  vmImage: ubuntu-latest
#  agent.name: 13900D

image image

image

image

Pool: [olxyz-self](https://dev.azure.com/obrienlabsxyz/0aca6b19-a922-4889-8b48-d7d59ec5c4d9/_settings/agentqueues?poolId=&queueId=19)
Agent: 13900D
Started: Just now
Duration: 3s

Job preparation parameters
ContinueOnError: False
TimeoutInMinutes: 60
CancelTimeoutInMinutes: 5
Expand:
  MaxConcurrency: 0
  ########## System Pipeline Decorator(s) ##########

  Begin evaluating template 'system-pre-steps.yml'
Evaluating: eq('true', variables['system.debugContext'])
Expanded: eq('true', Null)
Result: False
Evaluating: resources['repositories']['self']
Expanded: Object
Result: True
Evaluating: not(containsValue(job['steps']['*']['task']['id'], '6d15af64-176c-496d-b583-fd2ae21d4df4'))
Expanded: not(containsValue(Object, '6d15af64-176c-496d-b583-fd2ae21d4df4'))
Result: True
Evaluating: resources['repositories']['self']['checkoutOptions']
Result: Object
Evaluating: pair['key']
Result: 'fetchDepth'
Evaluating: pair['value']
Result: '1'
Finished evaluating template 'system-pre-steps.yml'
********************************************************************************
Template and static variable resolution complete. Final runtime YAML document:
steps:
- task: 6d15af64-176c-496d-b583-fd2ae21d4df4@1
  inputs:
    repository: self
    fetchDepth: 1

  MaxConcurrency: 0

Windows local agent

image

obriensystems commented 2 months ago

Test 0-bootstrap ADO trigger from 2nd org olapp

michael@cloudshell:~/tef-olapp$ ls
github
michael@cloudshell:~/tef-olapp$ mkdir ado
michael@cloudshell:~/tef-olapp$ cd ado
michael@cloudshell:~/tef-olapp/ado$ mkdir _obrienlabsxyz
michael@cloudshell:~/tef-olapp/ado$ cd _obrienlabsxyz/
michael@cloudshell:~/tef-olapp/ado/_obrienlabsxyz$ ls
michael@cloudshell:~/tef-olapp/ado/_obrienlabsxyz$ git clone https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding-fork/_git/pbmm-on-gcp-onboarding-fork
Cloning into 'pbmm-on-gcp-onboarding-fork'...
^C
michael@cloudshell:~/tef-olapp/ado/_obrienlabsxyz$ gcloud config set project tef-olapp
Updated property [core/project].
michael@cloudshell:~/tef-olapp/ado/_obrienlabsxyz (tef-olapp)$ ls
michael@cloudshell:~/tef-olapp/ado/_obrienlabsxyz (tef-olapp)$ git clone https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding-fork/_git/pbmm-on-gcp-onboarding-fork
Cloning into 'pbmm-on-gcp-onboarding-fork'...
Password for 'https://obrienlabsxyz@dev.azure.com': 
remote: Azure Repos
remote: Found 6401 objects to send. (52 ms)
Receiving objects:   0% (11/6401), 67.98 KiB | 48.00 KiB/s

it takes a while 1229-1235

Cloning into 'pbmm-on-gcp-onboarding-fork'...
Password for 'https://obrienlabsxyz@dev.azure.com': 
remote: Azure Repos
remote: Found 6401 objects to send. (52 ms)
Receiving objects:   0% (11/6401), 67.98 KiB | 48.00 KiB/s
error: RPC failed; curl 56 GnuTLS recv error (-54): Error in the pull function.
fetch-pack: unexpected disconnect while reading sideband packet
fatal: early EOF
fatal: fetch-pack: invalid index-pack output
michael@cloudshell:~/tef-olapp/ado/_obrienlabsxyz (tef-olapp)$

Try empty repo with a single pipeline - worked

https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/tef-gcp-pbmm-lz/_git/tef-gcp-pbmm-lz

michael@cloudshell:~/tef-olapp/ado/_obrienlabsxyz (tef-olapp)$ git clone https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/tef-gcp-pbmm-lz/_git/tef-gcp-pbmm-lz
Cloning into 'tef-gcp-pbmm-lz'...
Password for 'https://obrienlabsxyz@dev.azure.com': 
remote: Azure Repos
remote: Found 15 objects to send. (135 ms)
Unpacking objects: 100% (15/15), 2.33 KiB | 477.00 KiB/s, done.
michael@cloudshell:~/tef-olapp/ado/_obrienlabsxyz (tef-olapp)$ ls
tef-gcp-pbmm-lz
michael@cloudshell:~/tef-olapp/ad

retry clone from ado on pbmm repo - issue is 48KB/s

michael@cloudshell:~/tef-olapp/ado/_obrienlabsxyz (tef-olapp)$ git clone https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding-fork/_git/pbmm-on-gcp-onboarding-fork
Cloning into 'pbmm-on-gcp-onboarding-fork'...

Password for 'https://obrienlabsxyz@dev.azure.com': 
remote: Azure Repos
remote: Found 6401 objects to send. (76 ms)
Receiving objects:   0% (11/6401), 67.98 KiB | 48.00 KiB/s
error: RPC failed; curl 56 GnuTLS recv error (-54): Error in the pull function.
fetch-pack: unexpected disconnect while reading sideband packet
fatal: early EOF
fatal: fetch-pack: invalid index-pack output
obriensystems commented 2 months ago

Test GCP operation from the agent to start

in cloud shell prepare the SA

michael@cloudshell:~/tef-olapp/ado/_obrienlabsxyz (tef-olapp)$ gcloud iam service-accounts create ado-sa --display-name="ado-sa" --project=tef-olapp
Created service account [ado-sa].
michael@cloudshell:~/tef-olapp/ado/_obrienlabsxyz (tef-olapp)$ export PROJECT_ID=tef-olapp
michael@cloudshell:~/tef-olapp/ado/_obrienlabsxyz (tef-olapp)$ export SA_EMAIL=ado-sa@$PROJECT_ID.iam.gserviceaccount.com
michael@cloudshell:~/tef-olapp/ado/_obrienlabsxyz (tef-olapp)$ echo $SA_EMAIL
ado-sa@tef-olapp.iam.gserviceaccount.com
michael@cloudshell:~/tef-olapp/ado/_obrienlabsxyz (tef-olapp)$ gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:$SA_EMAIL --role roles/storage.admin --project=$PROJECT_ID
Updated IAM policy for project [tef-olapp].
bindings:
- members:
  - serviceAccount:153288813308@cloudbuild.gserviceaccount.com
  role: roles/cloudbuild.builds.builder
- members:
  - serviceAccount:service-153288813308@gcp-sa-cloudbuild.iam.gserviceaccount.com
  role: roles/cloudbuild.serviceAgent
- members:
  - serviceAccount:service-153288813308@containerregistry.iam.gserviceaccount.com
  role: roles/containerregistry.ServiceAgent
- members:
  - user:michael@obrienlabs.app
  role: roles/owner
- members:
  - serviceAccount:service-153288813308@gcp-sa-pubsub.iam.gserviceaccount.com
  role: roles/pubsub.serviceAgent
- members:
  - serviceAccount:ado-sa@tef-olapp.iam.gserviceaccount.com
  role: roles/storage.admin
etag: BwYW2urwbdQ=
version: 1
Screenshot 2024-04-24 at 13 23 50

generate a SA key

gcloud iam service-accounts keys create ado-sa.json --iam-account $SA_EMAIL --project=$PROJECT_ID
tr -d '\n' < ado-sa.json > ado-sa-oneline.json

Download key from GCP - upload to ADO | pipelines | library | secure files

Screenshot 2024-04-24 at 13 28 47 Screenshot 2024-04-24 at 13 30 10 Screenshot 2024-04-24 at 13 30 47

create service connection in project settings | pipelines | service connections

Screenshot 2024-04-24 at 13 36 04

generic

no -

Authorize the key for all pipelines - pipeline permissions tab

Screenshot 2024-04-24 at 13 38 38

try https://dev.to/ib1/azure-devops-recipe-deploying-google-cloud-function-to-gcp-22l3

trigger:
- main

pool: olxyz-self
#  vmImage: ubuntu-latest
#  agent.name: 13900D

steps:
- task: DownloadSecureFile@1
  name: authkey
  displayName: 'Download Service Account Key'
  inputs:
    secureFile: 'ado-sa-oneline.json'
    retryCount: '2'

- script: echo Hello, world!
  displayName: 'Run a one-line script'

- script: |
    echo Add other tasks to build, test, and deploy your project.
    echo See https://aka.ms/yaml
  displayName: 'Run a multi-line script'

on agent

2024-04-24 17:44:12Z: Running job: Job
2024-04-24 17:44:23Z: Job Job completed with result: Succeeded

task didnt run as it was not referenced in a step
2024-04-24T17:44:17.7234209Z ##[section]Finishing: Checkout tef-gcp-pbmm-lz@main to s
2024-04-24T17:44:17.7240632Z ##[section]Starting: Run a one-line script
2024-04-24T17:44:17.7242946Z ==============================================================================
2024-04-24T17:44:17.7242984Z Task         : Command line
2024-04-24T17:44:17.7243007Z Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2024-04-24T17:44:17.7243044Z Version      : 2.237.1
2024-04-24T17:44:17.7243064Z Author       : Microsoft Corporation
2024-04-24T17:44:17.7243088Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2024-04-24T17:44:17.7243309Z ==============================================================================
2024-04-24T17:44:18.5923224Z Generating script.
2024-04-24T17:44:18.5967591Z Script contents: shell
2024-04-24T17:44:18.5977126Z echo Hello, world!
2024-04-24T17:44:18.6100357Z ========================== Starting Command Output ===========================
2024-04-24T17:44:18.6221313Z ##[command]"C:\Windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "C:\opt\agent\_work\_temp\6730bfef-6b37-49d8-85eb-ddf07541d350.cmd""
2024-04-24T17:44:18.6408689Z Hello, world!
2024-04-24T17:44:18.6604951Z ##[section]Finishing: Run a one-line script
2024-04-24T17:44:18.6611479Z ##[section]Starting: Run a multi-line script
2024-04-24T17:44:18.6613943Z ==============================================================================
2024-04-24T17:44:18.6613981Z Task         : Command line
2024-04-24T17:44:18.6614003Z Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2024-04-24T17:44:18.6614038Z Version      : 2.237.1
2024-04-24T17:44:18.6614057Z Author       : Microsoft Corporation
2024-04-24T17:44:18.6614086Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2024-04-24T17:44:18.6614118Z ==============================================================================
2024-04-24T17:44:19.0442488Z Generating script.
2024-04-24T17:44:19.0580168Z ========================== Starting Command Output ===========================
2024-04-24T17:44:19.0691212Z ##[command]"C:\Windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "C:\opt\agent\_work\_temp\ce04b018-17b9-446c-a1e4-5d76bdd021b9.cmd""
2024-04-24T17:44:19.0828050Z Add other tasks to build, test, and deploy your project.
2024-04-24T17:44:19.0828659Z See https://aka.ms/yaml
2024-04-24T17:44:19.0968065Z ##[section]Finishing: Run a multi-line script
2024-04-24T17:44:19.0974600Z ##[section]Starting: Checkout tef-gcp-pbmm-lz@main to s
2024-04-24T17:44:19.0975994Z ==============================================================================
2024-04-24T17:44:19.0976032Z Task         : Get sources
2024-04-24T17:44:19.0976053Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2024-04-24T17:44:19.0976085Z Version      : 1.0.0
2024-04-24T17:44:19.0976109Z Author       : Microsoft
2024-04-24T17:44:19.0976130Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2024-04-24T17:44:19.0976159Z ==============================================================================
2024-04-24T17:44:19.2871436Z Cleaning any cached credential from repository: tef-gcp-pbmm-lz (Git)
2024-04-24T17:44:19.2917545Z ##[section]Finishing: Checkout tef-gcp-pbmm-lz@main to s

Add gcloud install

- script: |
    wget https://dl.google.com/dl/cloudsdk/release/google-cloud-sdk.tar.gz
    tar zxvf google-cloud-sdk.tar.gz && ./google-cloud-sdk/install.sh --quiet --usage-reporting=false --path-update=true
    PATH="google-cloud-sdk/bin:${PATH}"
    gcloud --quiet components update
  displayName: 'install gcloud SDK'

we need a windows version https://cloud.google.com/sdk/docs/install#windows

obriensystems commented 2 months ago

2 day turnaround on optional Azure based agents for form

Screenshot 2024-04-25 at 12 00 04
obriensystems commented 2 months ago

ADO pipeline yams

ado pipeline yamls will go into the following directory

obriensystems commented 2 months ago

testing in obrienlabs.app - 2nd deployment

michael@cloudshell:~/tef-olapp/ado/github (tef-olapp)$ git clone https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git
Cloning into 'pbmm-on-gcp-onboarding'...
remote: Enumerating objects: 6405, done.
remote: Counting objects: 100% (2592/2592), done.
remote: Compressing objects: 100% (952/952), done.
remote: Total 6405 (delta 1706), reused 2333 (delta 1603), pack-reused 3813
Receiving objects: 100% (6405/6405), 31.81 MiB | 27.49 MiB/s, done.
Resolving deltas: 100% (3900/3900), done.
obriensystems commented 2 months ago

Dockerfile image in DockerHub iteration in https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/409

retest on terraform 1.3.10

CONTAINER_IMAGE=terraform-example-foundation-ado
RELEASE_ID=0.0.1
DOCKER_FILE=Dockerfile
DOCKERHUB_ORG=obrienlabs
BUILD_ID=10001
BUILD_DIR=builds
mkdir $BUILD_DIR
TARGET_DIR=$BUILD_DIR/$BUILD_ID
mkdir $TARGET_DIR

# DockerHub only
docker build --rm=true --no-cache --build-arg build-id=$BUILD_ID -t $DOCKERHUB_ORG/$CONTAINER_IMAGE -f $DOCKER_FILE .
docker tag $DOCKERHUB_ORG/$CONTAINER_IMAGE $DOCKERHUB_ORG/$CONTAINER_IMAGE:$RELEASE_ID
docker tag $DOCKERHUB_ORG/$CONTAINER_IMAGE $DOCKERHUB_ORG/$CONTAINER_IMAGE:latest
docker push obrienlabs/$CONTAINER_IMAGE:$RELEASE_ID
docker push obrienlabs/$CONTAINER_IMAGE:latest

docker run obrienlabs/terraform-example-foundation-ado --version

The push refers to repository [docker.io/obrienlabs/terraform-example-foundation-ado]
fe0fec718258: Layer already exists 
600d3bb9c25f: Layer already exists 
5a3af3daf7a4: Layer already exists 
3a53c526edc7: Layer already exists 
84ff92691f90: Layer already exists 
32bfd004bf31: Layer already exists 
404802fd10b7: Layer already exists 
latest: digest: sha256:51f797316b959bf66785a183f4fa752d1157dfe3329336fe4bdde53f1be15672 size: 1786
test a terraform exe run on ia64 only - not arm64
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Terraform v1.3.10
on linux_amd64

retest remotely

michael@cloudshell:~ (tef-olxyz)$ docker image rm  obrienlabs/terraform-example-foundation-ado:latest -f
Untagged: obrienlabs/terraform-example-foundation-ado:latest
Untagged: obrienlabs/terraform-example-foundation-ado@sha256:f79fc4937575f143ddc29ca8958f0bf0dd67153587fbc6d05b5379cc16cb2205
Deleted: sha256:075b31b3e45cc809a55d9a753483febb99565ac12ef5bc020279a31e3ee74587
michael@cloudshell:~ (tef-olxyz)$ docker run --name tef-ado obrienlabs/terraform-example-foundation-ado:latest --version
Unable to find image 'obrienlabs/terraform-example-foundation-ado:latest' locally
latest: Pulling from obrienlabs/terraform-example-foundation-ado
d2aa1518a716: Already exists 
76ff73118202: Already exists 
3c2cba919283: Already exists 
b32fb4fcf463: Already exists 
af8c1adcdaeb: Already exists 
f31ad5bb9a07: Already exists 
9a6d62f9bf23: Pull complete 
Digest: sha256:c8a2438c1c6729cf1cd0ab5777bd537757655c6f209137af33b105c17721f369
Status: Downloaded newer image for obrienlabs/terraform-example-foundation-ado:latest
docker: Error response from daemon: Conflict. The container name "/tef-ado" is already in use by container "460dfe4001dd00d33ae6ccd3272adaf7779478dc1ee5f57359bc5ab41eab9830". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
michael@cloudshell:~ (tef-olxyz)$ docker ps -a
CONTAINER ID   IMAGE          COMMAND                 CREATED          STATUS                      PORTS     NAMES
460dfe4001dd   075b31b3e45c   "terraform --version"   15 minutes ago   Exited (0) 15 minutes ago             tef-ado
michael@cloudshell:~ (tef-olxyz)$ docker rm -v tef-ado
tef-ado
michael@cloudshell:~ (tef-olxyz)$ docker run --name tef-ado obrienlabs/terraform-example-foundation-ado:latest --version
Terraform v1.3.10
on linux_amd64

Your version of Terraform is out of date! The latest version
is 1.8.2. You can update by downloading from https://www.terraform.io/downloads.html
michael@cloudshell:~ (tef-olxyz)$
obriensystems commented 2 months ago

Test terraform 1.3.10 image in Azure Devops

Screenshot 2024-04-27 at 18 49 14

via azure-pipeline.yml

trigger:
- main

pool:
  vmImage: ubuntu-latest
- script: |
    docker run obrienlabs/terraform-example-foundation-ado --version
Screenshot 2024-04-27 at 20 15 11

replaces the artifact registry container build using tf-cloudbuilder

Screenshot 2024-04-27 at 20 30 04
obriensystems commented 2 months ago

terraform output reference for 0-bootstrap - relevant to ADO

7 repos (including one to build the docker container)

bootstrap_step_terraform_service_account_email = "sa-terraform-bootstrap@prj-b-seed-8919.iam.gserviceaccount.com"
organization_step_terraform_service_account_email = "sa-terraform-org@prj-b-seed-8919.iam.gserviceaccount.com"
projects_step_terraform_service_account_email = "sa-terraform-proj@prj-b-seed-8919.iam.gserviceaccount.com"
environment_step_terraform_service_account_email = "sa-terraform-env@prj-b-seed-8919.iam.gserviceaccount.com"

# not required except for local agents
cloud_build_peered_network_id = "projects/prj-b-cicd-82vv/global/networks/vpc-b-cbpools"
cloud_build_private_worker_pool_id = "projects/prj-b-cicd-82vv/locations/us-central1/workerPools/private-pool-yqvb"
cloud_build_worker_peered_ip_range = "192.168.0.0/24"
cloud_build_worker_range_id = "projects/prj-b-cicd-82vv/global/addresses/ga-b-cbpools-worker-pool-range"
cloud_builder_artifact_repo = "projects/prj-b-cicd-82vv/locations/us-central1/repositories/tf-runners"

cloudbuild_project_id = "prj-b-cicd-82vv"
seed_project_id = "prj-b-seed-8919"
# keep on GCP
gcs_bucket_tfstate = "bkt-prj-b-seed-tfstate-7120"
# keep on GCP
projects_gcs_bucket_tfstate = "bkt-prj-b-seed-8919-gcp-projects-tfstate"

# these repos will be on ADO - remove from GCP
csr_repos = {
  "gcp-bootstrap" = {
    "id" = "projects/prj-b-cicd-82vv/repos/gcp-bootstrap"
    "name" = "gcp-bootstrap"
    "project" = "prj-b-cicd-82vv"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-82vv/r/gcp-bootstrap"
  }
  "gcp-environments" = {
    "id" = "projects/prj-b-cicd-82vv/repos/gcp-environments"
    "name" = "gcp-environments"
    "project" = "prj-b-cicd-82vv"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-82vv/r/gcp-environments"
  }
  "gcp-networks" = {
    "id" = "projects/prj-b-cicd-82vv/repos/gcp-networks"
    "name" = "gcp-networks"
    "project" = "prj-b-cicd-82vv"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-82vv/r/gcp-networks"
  }
  "gcp-org" = {
    "id" = "projects/prj-b-cicd-82vv/repos/gcp-org"
    "name" = "gcp-org"
    "project" = "prj-b-cicd-82vv"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-82vv/r/gcp-org"
  }
  "gcp-policies" = {
    "id" = "projects/prj-b-cicd-82vv/repos/gcp-policies"
    "name" = "gcp-policies"
    "project" = "prj-b-cicd-82vv"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-82vv/r/gcp-policies"
  }
  "gcp-projects" = {
    "id" = "projects/prj-b-cicd-82vv/repos/gcp-projects"
    "name" = "gcp-projects"
    "project" = "prj-b-cicd-82vv"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-82vv/r/gcp-projects"
  }
  "tf-cloudbuilder" = {
    "id" = "projects/prj-b-cicd-82vv/repos/tf-cloudbuilder"
    "name" = "tf-cloudbuilder"
    "project" = "prj-b-cicd-82vv"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-82vv/r/tf-cloudbuilder"
  }
}

# investigate
gcs_bucket_cloudbuild_artifacts = {
  "bootstrap" = "bkt-prj-b-cicd-82vv-gcp-bootstrap-build-artifacts"
  "env" = "bkt-prj-b-cicd-82vv-gcp-environments-build-artifacts"
  "net" = "bkt-prj-b-cicd-82vv-gcp-networks-build-artifacts"
  "org" = "bkt-prj-b-cicd-82vv-gcp-org-build-artifacts"
  "proj" = "bkt-prj-b-cicd-82vv-gcp-projects-build-artifacts"
}

# these will be on the ADO org - can be removed from GCP
gcs_bucket_cloudbuild_logs = {
  "bootstrap" = "bkt-prj-b-cicd-82vv-gcp-bootstrap-build-logs"
  "env" = "bkt-prj-b-cicd-82vv-gcp-environments-build-logs"
  "net" = "bkt-prj-b-cicd-82vv-gcp-networks-build-logs"
  "org" = "bkt-prj-b-cicd-82vv-gcp-org-build-logs"
  "proj" = "bkt-prj-b-cicd-82vv-gcp-projects-build-logs"
}
obriensystems commented 2 months ago

testing in obrienlabs.app - 2nd deployment - local mbp m1max

While writing https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/gh399-ado/0-bootstrap/README-Azure-DevOps.md shadow steps https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/gh399-ado/0-bootstrap/README.md#deploying-with-cloud-build

Create new project ado

Screenshot 2024-04-29 at 11 04 55

repos | import

Screenshot 2024-04-29 at 11 09 46

new url is https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding

michaelobrien@mbp7 olapp % mkdir _deploy_test_399_from_ado
michaelobrien@mbp7 olapp % cd  _deploy_test_399_from_ado
michaelobrien@mbp7 _deploy_test_399_from_ado % git clone https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding
Cloning into 'pbmm-on-gcp-onboarding'...
remote: Azure Repos
remote: Found 6464 objects to send. (32 ms)
Receiving objects: 100% (6464/6464), 31.92 MiB | 17.82 MiB/s, done.
Resolving deltas: 100% (3929/3929), done.

downgrade terraform

Switch to the main branch - or a branch under active development

git checkout main

Generate GIT Credentials on the ADO repo

Clone ADO repo into your local environment

# replace YOUR-ORG with your ado organization
git clone https://YOUR-ORG@dev.azure.com/YOUR-ORG/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding
cd pbmm-on-gcp-onboarding/0-bootstrap

Rename terraform.example.tfvars to terraform.tfvars and update the file with values from your environment:

mv terraform.example.tfvars terraform.tfvars

Optionally: Use the helper script validate-requirements.sh to validate your environment:

Optionally: Downgrade Terraform to 1.3.10

obriensystems commented 2 months ago

ADO 0-bootstrap continued

6 repo creation

see reference https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/main/0-bootstrap/scripts/git_create_branches_helper.sh

Comment out sections in output.tf and variables.tf

uncomment sections in terraform.example.tfvars

uncomment sections in providers.tf - for any tf deploying ado infra

obriensystems commented 2 months ago

ADO 0-bootstrap continued - local gcloud shell instructions working

see

michaelobrien@mbp7 pbmm-on-gcp-onboarding % gcloud config set project tef-olapp                        
Updated property [core/project].
michaelobrien@mbp7 pbmm-on-gcp-onboarding % pwd
/Users/michaelobrien/wse_github/GoogleCloudPlatform/olapp/_deploy_test_399_from_ado/pbmm-on-gcp-onboarding

at step https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/gh399-ado/0-bootstrap/README-Azure-DevOps.md#rename-terraformexampletfvars-to-terraformtfvars-and-update-the-file-with-values-from-your-environment

reference: https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/gh399-ado/0-bootstrap/README-GitHub.md

editing: https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/edit/gh399-ado/0-bootstrap/README-Azure-DevOps.md

Clone the public ADO repository into your local environment

For local gcloud environment authentication setup - see https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/wiki/DevOps#authenticate-a-local-cloud-shell

# replace YOUR-ORG with your ado organization
git clone https://YOUR-ORG@dev.azure.com/YOUR-ORG/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding
cd pbmm-on-gcp-onboarding/0-bootstrap

Create 5 additional private GCP repos below

see Repos / Files / Dropdown

Screenshot 2024-04-29 at 12 18 26

gcp-bootstrap, gcp-policies, gcp-organization, gcp-networks, gcp-projects

Screenshot 2024-04-30 at 12 46 03 Screenshot 2024-04-30 at 12 46 30

gcp-bootstrap

  1. Clone the private gcp-bootstrap repository you created to host the 0-bootstrap terraform configuration at the same level of the pbmm-on-gcp-onboarding folder.

local gcloud example

michaelobrien@mbp7 _deploy_test_399_from_ado % git clone https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/gcp-bootstrap gcp-bootstrap
Cloning into 'gcp-bootstrap'...
remote: Azure Repos
remote: Found 3 objects to send. (23 ms)
Unpacking objects: 100% (3/3), 736 bytes | 368.00 KiB/s, done.
michaelobrien@mbp7 _deploy_test_399_from_ado % cd gcp-bootstrap 
michaelobrien@mbp7 gcp-bootstrap % ls
README.md
michaelobrien@mbp7 gcp-bootstrap % git status
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
obriensystems commented 2 months ago 
ichaelobrien@mbp7 gcp-bootstrap % git checkout -b plan
Switched to a new branch 'plan'
michaelobrien@mbp7 gcp-bootstrap % mkdir -p envs/shared
michaelobrien@mbp7 gcp-bootstrap % cp -RT ../pbmm-on-gcp-onboarding/0-bootstrap/ ./envs/shared
cp: illegal option -- T
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file target_file
       cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file ... target_directory
michaelobrien@mbp7 gcp-bootstrap % cp -RT ../pbmm-on-gcp-onboarding/0-bootstrap/ ./envs/shared
cp: illegal option -- T
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file target_file
       cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file ... target_directory
michaelobrien@mbp7 gcp-bootstrap % cp -R ../pbmm-on-gcp-onboarding/0-bootstrap/ ./envs/shared 
michaelobrien@mbp7 gcp-bootstrap % ls envs/shared 
Dockerfile          ado.tf.example          gitlab.tf.example       outputs.tf.local        terraform_cloud.tf.example
README-Azure-DevOps.md      backend.tf.cloud.example    groups.tf           provider.tf         variables.tf
README-GitHub.md        backend.tf.example      jenkins.tf.example      sa.tf               versions.tf
README-GitLab.md        backend.tf.local        main.tf             scripts
README-Jenkins.md       cb.tf.dont_use          modules             terraform-local.tf
README-Terraform-Cloud.md   files               onprem.md           terraform.example.tfvars
README.md           github.tf.example       outputs.tf.dont_use     terraform.mod.tfvars
michaelobrien@mbp7 gcp-bootstrap % cp -R ../pbmm-on-gcp-onboarding/policy-library/ ./policy-library
michaelobrien@mbp7 gcp-bootstrap % mkdir -p .github/workflows
michaelobrien@mbp7 gcp-bootstrap % cp ../pbmm-on-gcp-onboarding/build/github-tf-* ./.github/workflows/
michaelobrien@mbp7 gcp-bootstrap % cp ../pbmm-on-gcp-onboarding/build/tf-wrapper.sh .
michaelobrien@mbp7 gcp-bootstrap % chmod 755 ./tf-wrapper.sh
michaelobrien@mbp7 gcp-bootstrap % cd ./envs/shared
obriensystems commented 2 months ago

0-bootstrap ado continued

create 4 more repos via console gcp-policies, gcp-organization, gcp-networks, gcp-projects

Screenshot 2024-05-01 at 08 24 35

need to create an ADO version of the helper

./terraform-example-foundation/0-bootstrap/scripts/git_create_branches_helper.sh GITLAB

step 5 Seed the repository if it has not been initialized yet.

michaelobrien@mbp7 gcp-bootstrap % git commit --allow-empty -m 'repository seed'
[main 12cf71b] repository seed
michaelobrien@mbp7 gcp-bootstrap % git push --set-upstream origin main
Enumerating objects: 1, done.
Counting objects: 100% (1/1), done.
Writing objects: 100% (1/1), 195 bytes | 195.00 KiB/s, done.
Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Analyzing objects... (1/1) (21 ms)
remote: Validating commits... (1/1) done (8 ms)
remote: Storing packfile... done (62 ms)
remote: Storing index... done (59 ms)
To https://dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/gcp-bootstrap
   f15ef31..12cf71b  main -> main
branch 'main' set up to track 'origin/main'.
michaelobrien@mbp7 gcp-bootstrap % git checkout -b production
Switched to a new branch 'production'
michaelobrien@mbp7 gcp-bootstrap % git push --set-upstream origin production
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
To https://dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/gcp-bootstrap
 * [new branch]      production -> production
branch 'production' set up to track 'origin/production'.

ichaelobrien@mbp7 gcp-bootstrap % git checkout -b plan
Switched to a new branch 'plan'
michaelobrien@mbp7 gcp-bootstrap % mkdir -p envs/shared
michaelobrien@mbp7 gcp-bootstrap % cp -RT ../pbmm-on-gcp-onboarding/0-bootstrap/ ./envs/shared
cp: illegal option -- T
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file target_file
       cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file ... target_directory
michaelobrien@mbp7 gcp-bootstrap % cp -RT ../pbmm-on-gcp-onboarding/0-bootstrap/ ./envs/shared
cp: illegal option -- T
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file target_file
       cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file ... target_directory
michaelobrien@mbp7 gcp-bootstrap % cp -R ../pbmm-on-gcp-onboarding/0-bootstrap/ ./envs/shared 
michaelobrien@mbp7 gcp-bootstrap % ls envs/shared 
Dockerfile          ado.tf.example          gitlab.tf.example       outputs.tf.local        terraform_cloud.tf.example
README-Azure-DevOps.md      backend.tf.cloud.example    groups.tf           provider.tf         variables.tf
README-GitHub.md        backend.tf.example      jenkins.tf.example      sa.tf               versions.tf
README-GitLab.md        backend.tf.local        main.tf             scripts
README-Jenkins.md       cb.tf.dont_use          modules             terraform-local.tf
README-Terraform-Cloud.md   files               onprem.md           terraform.example.tfvars
README.md           github.tf.example       outputs.tf.dont_use     terraform.mod.tfvars
michaelobrien@mbp7 gcp-bootstrap % cp -R ../pbmm-on-gcp-onboarding/policy-library/ ./policy-library
michaelobrien@mbp7 gcp-bootstrap % mkdir -p .github/workflows
michaelobrien@mbp7 gcp-bootstrap % cp ../pbmm-on-gcp-onboarding/build/github-tf-* ./.github/workflows/
michaelobrien@mbp7 gcp-bootstrap % cp ../pbmm-on-gcp-onboarding/build/tf-wrapper.sh .
michaelobrien@mbp7 gcp-bootstrap % chmod 755 ./tf-wrapper.sh
michaelobrien@mbp7 gcp-bootstrap % cd ./envs/shared
obriensystems commented 2 months ago

See latest 3rd org testing 50 min for #421 in https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/421#issuecomment-2093522430

obriensystems commented 2 months ago

merge main into gh399-ado

ichaelobrien@mbp7 gh399-ado % git clone git@github.com:GoogleCloudPlatform/pbmm-on-gcp-onboarding.git
Cloning into 'pbmm-on-gcp-onboarding'...
remote: Enumerating objects: 6563, done.
remote: Counting objects: 100% (2307/2307), done.
remote: Compressing objects: 100% (804/804), done.
remote: Total 6563 (delta 1599), reused 2004 (delta 1471), pack-reused 4256
Receiving objects: 100% (6563/6563), 31.81 MiB | 11.61 MiB/s, done.
Resolving deltas: 100% (4021/4021), done.
michaelobrien@mbp7 gh399-ado % git checkout gh399-ado
fatal: not a git repository (or any of the parent directories): .git
michaelobrien@mbp7 gh399-ado % cd pbmm-on-gcp-onboarding 
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git checkout gh399-ado   
branch 'gh399-ado' set up to track 'origin/gh399-ado'.
Switched to a new branch 'gh399-ado'
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git merge main
Auto-merging 0-bootstrap/README.md
Auto-merging 0-bootstrap/terraform.example.tfvars
Merge made by the 'ort' strategy.
 0-bootstrap/README.md                                          | 38 ++++++++++++++++++++++++++++++++++++--
 0-bootstrap/{cb.tf.dont_use => cb.tf}                          |  0
 0-bootstrap/{outputs.tf.dont_use => outputs.tf}                |  8 ++++++++
 0-bootstrap/{terraform-local.tf => terraform-local.tf.example} | 16 ++++++++++++++++
 0-bootstrap/terraform.example.tfvars                           | 35 ++++++++++++++++++-----------------
 0-bootstrap/terraform.mod.tfvars                               | 63 ---------------------------------------------------------------
 1-org/envs/shared/terraform.example.tfvars                     | 15 ++++++---------
 1-org/envs/shared/terraform.mod.tfvars                         | 37 -------------------------------------
 2-environments/terraform.example.tfvars                        |  2 +-
 2-environments/terraform.mod.tfvars                            | 18 ------------------
 3-networks-dual-svpc/access_context.auto.mod.tfvars            | 17 -----------------
 3-networks-dual-svpc/common.auto.example.tfvars                | 11 +++++++----
 3-networks-dual-svpc/common.auto.mod.tfvars                    | 28 ----------------------------
 3-networks-dual-svpc/shared.auto.mod.tfvars                    | 28 ----------------------------
 3-networks-hub-and-spoke/access_context.auto.mod.tfvars        | 17 -----------------
 3-networks-hub-and-spoke/common.auto.example.tfvars            |  5 +++--
 3-networks-hub-and-spoke/common.auto.mod.tfvars                | 28 ----------------------------
 3-networks-hub-and-spoke/shared.auto.mod.tfvars                | 28 ----------------------------
 4-projects/common.auto.mod.tfvars                              | 17 -----------------
 4-projects/development.auto.example.tfvars                     |  6 ++++++
 4-projects/development.auto.mod.tfvars                         | 18 ------------------
 4-projects/non-production.auto.example.tfvars                  |  6 ++++++
 4-projects/non-production.auto.mod.tfvars                      | 18 ------------------
 4-projects/production.auto.example.tfvars                      |  6 ++++++
 4-projects/production.auto.mod.tfvars                          | 18 ------------------
 4-projects/shared.auto.example.tfvars                          |  3 ++-
 4-projects/shared.auto.mod.tfvars                              | 18 ------------------
 README.md                                                      | 16 ++++++++++++++++
 helpers/foundation-deployer/go.mod                             |  2 +-
 helpers/foundation-deployer/go.sum                             |  8 ++++----
 30 files changed, 136 insertions(+), 394 deletions(-)
 rename 0-bootstrap/{cb.tf.dont_use => cb.tf} (100%)
 rename 0-bootstrap/{outputs.tf.dont_use => outputs.tf} (96%)
 rename 0-bootstrap/{terraform-local.tf => terraform-local.tf.example} (54%)
 delete mode 100644 0-bootstrap/terraform.mod.tfvars
 delete mode 100644 1-org/envs/shared/terraform.mod.tfvars
 delete mode 100644 2-environments/terraform.mod.tfvars
 delete mode 100644 3-networks-dual-svpc/access_context.auto.mod.tfvars
 delete mode 100644 3-networks-dual-svpc/common.auto.mod.tfvars
 delete mode 100644 3-networks-dual-svpc/shared.auto.mod.tfvars
 delete mode 100644 3-networks-hub-and-spoke/access_context.auto.mod.tfvars
 delete mode 100644 3-networks-hub-and-spoke/common.auto.mod.tfvars
 delete mode 100644 3-networks-hub-and-spoke/shared.auto.mod.tfvars
 delete mode 100644 4-projects/common.auto.mod.tfvars
 delete mode 100644 4-projects/development.auto.mod.tfvars
 delete mode 100644 4-projects/non-production.auto.mod.tfvars
 delete mode 100644 4-projects/production.auto.mod.tfvars
 delete mode 100644 4-projects/shared.auto.mod.tfvars
obriensystems commented 2 months ago

rebase ADO from github

michaelobrien@mbp7 pbmm-on-gcp-onboarding % git status
On branch gh399-ado
Your branch is up to date with 'origin/gh399-ado'.

nothing to commit, working tree clean
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git checkout main
branch 'main' set up to track 'origin/main'.
Switched to a new branch 'main'
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git pull
Already up to date.
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git remote add upstream https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git 
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git fetch upstream
remote: Enumerating objects: 102, done.
remote: Counting objects: 100% (102/102), done.
remote: Compressing objects: 100% (56/56), done.
remote: Total 102 (delta 55), reused 87 (delta 44), pack-reused 0
Receiving objects: 100% (102/102), 88.98 KiB | 2.22 MiB/s, done.
Resolving deltas: 100% (55/55), completed with 2 local objects.
From https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding
 * [new branch]      20240504-main-cb-ready                                         -> upstream/20240504-main-cb-ready
 * [new branch]      243-tef-retrofit                                               -> upstream/243-tef-retrofit
 * [new branch]      318-log-sink-alerting                                          -> upstream/318-log-sink-alerting
 * [new branch]      332-dev-prov-client-v20230917                                  -> upstream/332-dev-prov-client-v20230917
 * [new branch]      341-labels-off-332                                             -> upstream/341-labels-off-332
 * [new branch]      341-tags-off-332                                               -> upstream/341-tags-off-332
 * [new branch]      345-landing-zone-fortigate-light                               -> upstream/345-landing-zone-fortigate-light
 * [new branch]      351-landing-zone-fortigate-cloud-setup                         -> upstream/351-landing-zone-fortigate-cloud-setup
 * [new branch]      craigenator                                                    -> upstream/craigenator
 * [new branch]      craigenator-network                                            -> upstream/craigenator-network
 * [new branch]      dan-fix-version-upgrade                                        -> upstream/dan-fix-version-upgrade
 * [new branch]      dependabot/go_modules/helpers/foundation-deployer/github.com/hashicorp/go-getter-1.7.4 -> upstream/dependabot/go_modules/helpers/foundation-deployer/github.com/hashicorp/go-getter-1.7.4
 * [new branch]      dependabot/go_modules/helpers/foundation-deployer/golang.org/x/net-0.23.0 -> upstream/dependabot/go_modules/helpers/foundation-deployer/golang.org/x/net-0.23.0
 * [new branch]      dependabot/go_modules/test/integration/github.com/hashicorp/go-getter-1.7.4 -> upstream/dependabot/go_modules/test/integration/github.com/hashicorp/go-getter-1.7.4
 * [new branch]      dependabot/go_modules/test/integration/golang.org/x/net-0.23.0 -> upstream/dependabot/go_modules/test/integration/golang.org/x/net-0.23.0
 * [new branch]      example                                                        -> upstream/example
 * [new branch]      fmichaelobrien                                                 -> upstream/fmichaelobrien
 * [new branch]      fmichaelobrien-example-nonprod                                 -> upstream/fmichaelobrien-example-nonprod
 * [new branch]      fmichaelobrien-example-prod                                    -> upstream/fmichaelobrien-example-prod
 * [new branch]      fmichaelobrien-pr-113                                          -> upstream/fmichaelobrien-pr-113
 * [new branch]      fortigate-yopps                                                -> upstream/fortigate-yopps
 * [new branch]      gh357-tef-v4-fork                                              -> upstream/gh357-tef-v4-fork
 * [new branch]      gh360-day0-deploy-example                                      -> upstream/gh360-day0-deploy-example
 * [new branch]      gh362-bootstrap-prereq                                         -> upstream/gh362-bootstrap-prereq
 * [new branch]      gh377-architecture                                             -> upstream/gh377-architecture
 * [new branch]      gh387-upstream-sync                                            -> upstream/gh387-upstream-sync
 * [new branch]      gh399-ado                                                      -> upstream/gh399-ado
 * [new branch]      hotfix/cloudbuildjob                                           -> upstream/hotfix/cloudbuildjob
 * [new branch]      hotfix/guardrailsiam                                           -> upstream/hotfix/guardrailsiam
 * [new branch]      main                                                           -> upstream/main
 * [new branch]      pr-mro-bootstrap-localization                                  -> upstream/pr-mro-bootstrap-localization
 * [new branch]      pr-mro-bootstrap-run-terraform-local-041524                    -> upstream/pr-mro-bootstrap-run-terraform-local-041524
 * [new branch]      pr-mro-tef-fix-bucket-name-length-041524                       -> upstream/pr-mro-tef-fix-bucket-name-length-041524
 * [new branch]      pr-mro-tef-multi-instance-041524                               -> upstream/pr-mro-tef-multi-instance-041524
 * [new branch]      tm-lz-org                                                      -> upstream/tm-lz-org
 * [new branch]      v020                                                           -> upstream/v020
 * [new branch]      v20240407                                                      -> upstream/v20240407
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git merge upstream/main main
Updating fc4b948..b209766
Fast-forward
 0-bootstrap/README.md                                          | 38 ++++++++++++++++++++++++++++++++++++--
 0-bootstrap/{cb.tf.dont_use => cb.tf}                          |  0
 0-bootstrap/{outputs.tf.dont_use => outputs.tf}                |  8 ++++++++
 0-bootstrap/{terraform-local.tf => terraform-local.tf.example} | 16 ++++++++++++++++
 0-bootstrap/terraform.example.tfvars                           | 35 ++++++++++++++++++-----------------
 0-bootstrap/terraform.mod.tfvars                               | 63 ---------------------------------------------------------------
 1-org/envs/shared/terraform.example.tfvars                     | 15 ++++++---------
 1-org/envs/shared/terraform.mod.tfvars                         | 37 -------------------------------------
 2-environments/terraform.example.tfvars                        |  2 +-
 2-environments/terraform.mod.tfvars                            | 18 ------------------
 3-networks-dual-svpc/access_context.auto.mod.tfvars            | 17 -----------------
 3-networks-dual-svpc/common.auto.example.tfvars                | 11 +++++++----
 3-networks-dual-svpc/common.auto.mod.tfvars                    | 28 ----------------------------
 3-networks-dual-svpc/shared.auto.mod.tfvars                    | 28 ----------------------------
 3-networks-hub-and-spoke/access_context.auto.mod.tfvars        | 17 -----------------
 3-networks-hub-and-spoke/common.auto.example.tfvars            |  5 +++--
 3-networks-hub-and-spoke/common.auto.mod.tfvars                | 28 ----------------------------
 3-networks-hub-and-spoke/shared.auto.mod.tfvars                | 28 ----------------------------
 4-projects/common.auto.mod.tfvars                              | 17 -----------------
 4-projects/development.auto.example.tfvars                     |  6 ++++++
 4-projects/development.auto.mod.tfvars                         | 18 ------------------
 4-projects/non-production.auto.example.tfvars                  |  6 ++++++
 4-projects/non-production.auto.mod.tfvars                      | 18 ------------------
 4-projects/production.auto.example.tfvars                      |  6 ++++++
 4-projects/production.auto.mod.tfvars                          | 18 ------------------
 4-projects/shared.auto.example.tfvars                          |  3 ++-
 4-projects/shared.auto.mod.tfvars                              | 18 ------------------
 README.md                                                      | 16 ++++++++++++++++
 helpers/foundation-deployer/go.mod                             |  2 +-
 helpers/foundation-deployer/go.sum                             |  8 ++++----
 30 files changed, 136 insertions(+), 394 deletions(-)
 rename 0-bootstrap/{cb.tf.dont_use => cb.tf} (100%)
 rename 0-bootstrap/{outputs.tf.dont_use => outputs.tf} (96%)
 rename 0-bootstrap/{terraform-local.tf => terraform-local.tf.example} (54%)
 delete mode 100644 0-bootstrap/terraform.mod.tfvars
 delete mode 100644 1-org/envs/shared/terraform.mod.tfvars
 delete mode 100644 2-environments/terraform.mod.tfvars
 delete mode 100644 3-networks-dual-svpc/access_context.auto.mod.tfvars
 delete mode 100644 3-networks-dual-svpc/common.auto.mod.tfvars
 delete mode 100644 3-networks-dual-svpc/shared.auto.mod.tfvars
 delete mode 100644 3-networks-hub-and-spoke/access_context.auto.mod.tfvars
 delete mode 100644 3-networks-hub-and-spoke/common.auto.mod.tfvars
 delete mode 100644 3-networks-hub-and-spoke/shared.auto.mod.tfvars
 delete mode 100644 4-projects/common.auto.mod.tfvars
 delete mode 100644 4-projects/development.auto.mod.tfvars
 delete mode 100644 4-projects/non-production.auto.mod.tfvars
 delete mode 100644 4-projects/production.auto.mod.tfvars
 delete mode 100644 4-projects/shared.auto.mod.tfvars

michaelobrien@mbp7 pbmm-on-gcp-onboarding % git push origin main
Enumerating objects: 75, done.
Counting objects: 100% (75/75), done.
Delta compression using up to 10 threads
Compressing objects: 100% (28/28), done.
Writing objects: 100% (51/51), 15.80 KiB | 15.80 MiB/s, done.
Total 51 (delta 34), reused 39 (delta 22), pack-reused 0
remote: Analyzing objects... (51/51) (11 ms)
remote: Validating commits... (14/14) done (4 ms)
remote: Storing packfile... done (88 ms)
remote: Storing index... done (54 ms)
To https://dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding
   fc4b948..b209766  main -> main

michaelobrien@mbp7 pbmm-on-gcp-onboarding % git checkout gh399-ado      
Switched to branch 'gh399-ado'
Your branch is up to date with 'origin/gh399-ado'.
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git merge main
Auto-merging 0-bootstrap/README.md
Merge made by the 'ort' strategy.
 0-bootstrap/README.md                                          | 38 ++++++++++++++++++++++++++++++++++++--
 0-bootstrap/{cb.tf.dont_use => cb.tf}                          |  0
 0-bootstrap/{outputs.tf.dont_use => outputs.tf}                |  8 ++++++++
 0-bootstrap/{terraform-local.tf => terraform-local.tf.example} | 16 ++++++++++++++++
 0-bootstrap/terraform.example.tfvars                           | 35 ++++++++++++++++++-----------------
 0-bootstrap/terraform.mod.tfvars                               | 63 ---------------------------------------------------------------
 1-org/envs/shared/terraform.example.tfvars                     | 15 ++++++---------
 1-org/envs/shared/terraform.mod.tfvars                         | 37 -------------------------------------
 2-environments/terraform.example.tfvars                        |  2 +-
 2-environments/terraform.mod.tfvars                            | 18 ------------------
 3-networks-dual-svpc/access_context.auto.mod.tfvars            | 17 -----------------
 3-networks-dual-svpc/common.auto.example.tfvars                | 11 +++++++----
 3-networks-dual-svpc/common.auto.mod.tfvars                    | 28 ----------------------------
 3-networks-dual-svpc/shared.auto.mod.tfvars                    | 28 ----------------------------
 3-networks-hub-and-spoke/access_context.auto.mod.tfvars        | 17 -----------------
 3-networks-hub-and-spoke/common.auto.example.tfvars            |  5 +++--
 3-networks-hub-and-spoke/common.auto.mod.tfvars                | 28 ----------------------------
 3-networks-hub-and-spoke/shared.auto.mod.tfvars                | 28 ----------------------------
 4-projects/common.auto.mod.tfvars                              | 17 -----------------
 4-projects/development.auto.example.tfvars                     |  6 ++++++
 4-projects/development.auto.mod.tfvars                         | 18 ------------------
 4-projects/non-production.auto.example.tfvars                  |  6 ++++++
 4-projects/non-production.auto.mod.tfvars                      | 18 ------------------
 4-projects/production.auto.example.tfvars                      |  6 ++++++
 4-projects/production.auto.mod.tfvars                          | 18 ------------------
 4-projects/shared.auto.example.tfvars                          |  3 ++-
 4-projects/shared.auto.mod.tfvars                              | 18 ------------------
 README.md                                                      | 16 ++++++++++++++++
 helpers/foundation-deployer/go.mod                             |  2 +-
 helpers/foundation-deployer/go.sum                             |  8 ++++----
 30 files changed, 136 insertions(+), 394 deletions(-)
 rename 0-bootstrap/{cb.tf.dont_use => cb.tf} (100%)
 rename 0-bootstrap/{outputs.tf.dont_use => outputs.tf} (96%)
 rename 0-bootstrap/{terraform-local.tf => terraform-local.tf.example} (54%)
 delete mode 100644 0-bootstrap/terraform.mod.tfvars
 delete mode 100644 1-org/envs/shared/terraform.mod.tfvars
 delete mode 100644 2-environments/terraform.mod.tfvars
 delete mode 100644 3-networks-dual-svpc/access_context.auto.mod.tfvars
 delete mode 100644 3-networks-dual-svpc/common.auto.mod.tfvars
 delete mode 100644 3-networks-dual-svpc/shared.auto.mod.tfvars
 delete mode 100644 3-networks-hub-and-spoke/access_context.auto.mod.tfvars
 delete mode 100644 3-networks-hub-and-spoke/common.auto.mod.tfvars
 delete mode 100644 3-networks-hub-and-spoke/shared.auto.mod.tfvars
 delete mode 100644 4-projects/common.auto.mod.tfvars
 delete mode 100644 4-projects/development.auto.mod.tfvars
 delete mode 100644 4-projects/non-production.auto.mod.tfvars
 delete mode 100644 4-projects/production.auto.mod.tfvars
 delete mode 100644 4-projects/shared.auto.mod.tfvars
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git commit -m "#399 - upstream main merge"
On branch gh399-ado
Your branch is ahead of 'origin/gh399-ado' by 16 commits.
  (use "git push" to publish your local commits)

nothing to commit, working tree clean
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git push origin gh399-ado                 
Enumerating objects: 10, done.
Counting objects: 100% (10/10), done.
Delta compression using up to 10 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 695 bytes | 695.00 KiB/s, done.
Total 4 (delta 3), reused 0 (delta 0), pack-reused 0
remote: Analyzing objects... (4/4) (171 ms)
remote: Validating commits... (1/1) done (1 ms)
remote: Storing packfile... done (61 ms)
remote: Storing index... done (74 ms)
To https://dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding
   52b57c8..bb6d4e0  gh399-ado -> gh399-ado
Screenshot 2024-05-05 at 12 34 08 Screenshot 2024-05-05 at 12 34 36
obriensystems commented 2 months ago

restart 0-bootstrap ADO testing

Sync with main first on osx ado repo

ichaelobrien@mbp7 0-bootstrap % ls
Dockerfile          ado.tf.example          gitlab.tf.example       outputs.tf.local        terraform_cloud.tf.example
README-Azure-DevOps.md      backend.tf.cloud.example    groups.tf           provider.tf         variables.tf
README-GitHub.md        backend.tf.example      jenkins.tf.example      sa.tf               versions.tf
README-GitLab.md        backend.tf.local        main.tf             scripts
README-Jenkins.md       cb.tf.dont_use          modules             terraform-local.tf
README-Terraform-Cloud.md   files               onprem.md           terraform.example.tfvars
README.md           github.tf.example       outputs.tf.dont_use     terraform.mod.tfvars
michaelobrien@mbp7 0-bootstrap % git pull
remote: Azure Repos
remote: Found 61 objects to send. (2 ms)
Unpacking objects: 100% (61/61), 53.11 KiB | 1.47 MiB/s, done.
From https://dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding
   52b57c8..bb6d4e0  gh399-ado  -> origin/gh399-ado
   fc4b948..b209766  main       -> origin/main
Updating 52b57c8..bb6d4e0
Fast-forward
 0-bootstrap/README.md                                          | 38 ++++++++++++++++++++++++++++++++++++--
 0-bootstrap/{cb.tf.dont_use => cb.tf}                          |  0
 0-bootstrap/{outputs.tf.dont_use => outputs.tf}                |  8 ++++++++
 0-bootstrap/{terraform-local.tf => terraform-local.tf.example} | 16 ++++++++++++++++
 0-bootstrap/terraform.example.tfvars                           | 35 ++++++++++++++++++-----------------
 0-bootstrap/terraform.mod.tfvars                               | 63 ---------------------------------------------------------------
 1-org/envs/shared/terraform.example.tfvars                     | 15 ++++++---------
 1-org/envs/shared/terraform.mod.tfvars                         | 37 -------------------------------------
 2-environments/terraform.example.tfvars                        |  2 +-
 2-environments/terraform.mod.tfvars                            | 18 ------------------
 3-networks-dual-svpc/access_context.auto.mod.tfvars            | 17 -----------------
 3-networks-dual-svpc/common.auto.example.tfvars                | 11 +++++++----
 3-networks-dual-svpc/common.auto.mod.tfvars                    | 28 ----------------------------
 3-networks-dual-svpc/shared.auto.mod.tfvars                    | 28 ----------------------------
 3-networks-hub-and-spoke/access_context.auto.mod.tfvars        | 17 -----------------
 3-networks-hub-and-spoke/common.auto.example.tfvars            |  5 +++--
 3-networks-hub-and-spoke/common.auto.mod.tfvars                | 28 ----------------------------
 3-networks-hub-and-spoke/shared.auto.mod.tfvars                | 28 ----------------------------
 4-projects/common.auto.mod.tfvars                              | 17 -----------------
 4-projects/development.auto.example.tfvars                     |  6 ++++++
 4-projects/development.auto.mod.tfvars                         | 18 ------------------
 4-projects/non-production.auto.example.tfvars                  |  6 ++++++
 4-projects/non-production.auto.mod.tfvars                      | 18 ------------------
 4-projects/production.auto.example.tfvars                      |  6 ++++++
 4-projects/production.auto.mod.tfvars                          | 18 ------------------
 4-projects/shared.auto.example.tfvars                          |  3 ++-
 4-projects/shared.auto.mod.tfvars                              | 18 ------------------
 README.md                                                      | 16 ++++++++++++++++
 helpers/foundation-deployer/go.mod                             |  2 +-
 helpers/foundation-deployer/go.sum                             |  8 ++++----
 30 files changed, 136 insertions(+), 394 deletions(-)
 rename 0-bootstrap/{cb.tf.dont_use => cb.tf} (100%)
 rename 0-bootstrap/{outputs.tf.dont_use => outputs.tf} (96%)
 rename 0-bootstrap/{terraform-local.tf => terraform-local.tf.example} (54%)
 delete mode 100644 0-bootstrap/terraform.mod.tfvars
 delete mode 100644 1-org/envs/shared/terraform.mod.tfvars
 delete mode 100644 2-environments/terraform.mod.tfvars
 delete mode 100644 3-networks-dual-svpc/access_context.auto.mod.tfvars
 delete mode 100644 3-networks-dual-svpc/common.auto.mod.tfvars
 delete mode 100644 3-networks-dual-svpc/shared.auto.mod.tfvars
 delete mode 100644 3-networks-hub-and-spoke/access_context.auto.mod.tfvars
 delete mode 100644 3-networks-hub-and-spoke/common.auto.mod.tfvars
 delete mode 100644 3-networks-hub-and-spoke/shared.auto.mod.tfvars
 delete mode 100644 4-projects/common.auto.mod.tfvars
 delete mode 100644 4-projects/development.auto.mod.tfvars
 delete mode 100644 4-projects/non-production.auto.mod.tfvars
 delete mode 100644 4-projects/production.auto.mod.tfvars
 delete mode 100644 4-projects/shared.auto.mod.tfvars

rebase from upstream specifically for gh399-ado not just a main merge into gh399-ado - missing changes in 399 for terraform.example.tf

https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/wiki/DevOps#pbmm-upstream---to-ado-repo

still pushing to github not ado

michaelobrien@mbp7 ado_olxyz % mkdir upstream 
michaelobrien@mbp7 ado_olxyz %  git clone git@github.com:GoogleCloudPlatform/pbmm-on-gcp-onboarding.git
Cloning into 'pbmm-on-gcp-onboarding'...
remote: Enumerating objects: 6572, done.
remote: Counting objects: 100% (2316/2316), done.
remote: Compressing objects: 100% (809/809), done.
remote: Total 6572 (delta 1604), reused 2012 (delta 1475), pack-reused 4256
Receiving objects: 100% (6572/6572), 31.81 MiB | 27.06 MiB/s, done.
Resolving deltas: 100% (4026/4026), done.
michaelobrien@mbp7 ado_olxyz % ls                        
pbmm-on-gcp-onboarding  resync          upstream
michaelobrien@mbp7 ado_olxyz % cd pbmm-on-gcp-onboarding 
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git status
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git checkout gh399-ado
branch 'gh399-ado' set up to track 'origin/gh399-ado'.
Switched to a new branch 'gh399-ado'
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git remote add upstream https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git fetch upstream                                                                       
From https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding
 * [new branch]      20240504-main-cb-ready                                         -> upstream/20240504-main-cb-ready
 * [new branch]      243-tef-retrofit                                               -> upstream/243-tef-retrofit
 * [new branch]      318-log-sink-alerting                                          -> upstream/318-log-sink-alerting
 * [new branch]      332-dev-prov-client-v20230917                                  -> upstream/332-dev-prov-client-v20230917
 * [new branch]      341-labels-off-332                                             -> upstream/341-labels-off-332
 * [new branch]      341-tags-off-332                                               -> upstream/341-tags-off-332
 * [new branch]      345-landing-zone-fortigate-light                               -> upstream/345-landing-zone-fortigate-light
 * [new branch]      351-landing-zone-fortigate-cloud-setup                         -> upstream/351-landing-zone-fortigate-cloud-setup
 * [new branch]      craigenator                                                    -> upstream/craigenator
 * [new branch]      craigenator-network                                            -> upstream/craigenator-network
 * [new branch]      dan-fix-version-upgrade                                        -> upstream/dan-fix-version-upgrade
 * [new branch]      dependabot/go_modules/helpers/foundation-deployer/github.com/hashicorp/go-getter-1.7.4 -> upstream/dependabot/go_modules/helpers/foundation-deployer/github.com/hashicorp/go-getter-1.7.4
 * [new branch]      dependabot/go_modules/helpers/foundation-deployer/golang.org/x/net-0.23.0 -> upstream/dependabot/go_modules/helpers/foundation-deployer/golang.org/x/net-0.23.0
 * [new branch]      dependabot/go_modules/test/integration/github.com/hashicorp/go-getter-1.7.4 -> upstream/dependabot/go_modules/test/integration/github.com/hashicorp/go-getter-1.7.4
 * [new branch]      dependabot/go_modules/test/integration/golang.org/x/net-0.23.0 -> upstream/dependabot/go_modules/test/integration/golang.org/x/net-0.23.0
 * [new branch]      example                                                        -> upstream/example
 * [new branch]      fmichaelobrien                                                 -> upstream/fmichaelobrien
 * [new branch]      fmichaelobrien-example-nonprod                                 -> upstream/fmichaelobrien-example-nonprod
 * [new branch]      fmichaelobrien-example-prod                                    -> upstream/fmichaelobrien-example-prod
 * [new branch]      fmichaelobrien-pr-113                                          -> upstream/fmichaelobrien-pr-113
 * [new branch]      fortigate-yopps                                                -> upstream/fortigate-yopps
 * [new branch]      gh357-tef-v4-fork                                              -> upstream/gh357-tef-v4-fork
 * [new branch]      gh360-day0-deploy-example                                      -> upstream/gh360-day0-deploy-example
 * [new branch]      gh362-bootstrap-prereq                                         -> upstream/gh362-bootstrap-prereq
 * [new branch]      gh377-architecture                                             -> upstream/gh377-architecture
 * [new branch]      gh387-upstream-sync                                            -> upstream/gh387-upstream-sync
 * [new branch]      gh399-ado                                                      -> upstream/gh399-ado
 * [new branch]      hotfix/cloudbuildjob                                           -> upstream/hotfix/cloudbuildjob
 * [new branch]      hotfix/guardrailsiam                                           -> upstream/hotfix/guardrailsiam
 * [new branch]      main                                                           -> upstream/main
 * [new branch]      pr-mro-bootstrap-localization                                  -> upstream/pr-mro-bootstrap-localization
 * [new branch]      pr-mro-bootstrap-run-terraform-local-041524                    -> upstream/pr-mro-bootstrap-run-terraform-local-041524
 * [new branch]      pr-mro-tef-fix-bucket-name-length-041524                       -> upstream/pr-mro-tef-fix-bucket-name-length-041524
 * [new branch]      pr-mro-tef-multi-instance-041524                               -> upstream/pr-mro-tef-multi-instance-041524
 * [new branch]      tm-lz-org                                                      -> upstream/tm-lz-org
 * [new branch]      v020                                                           -> upstream/v020
 * [new branch]      v20240407                                                      -> upstream/v20240407
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git merge upstream/gh399-ado gh399-ado                                                   
Already up to date.
michaelobrien@mbp7 pbmm-on-gcp-onboarding % vi 0-bootstrap/terraform.example.tfvars 
michaelobrien@mbp7 pbmm-on-gcp-onboarding % # ado code already there
zsh: command not found: #
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git status
On branch gh399-ado
Your branch is up to date with 'origin/gh399-ado'.

nothing to commit, working tree clean
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git push origin gh399-ado
Everything up-to-date
michaelobrien@mbp7 pbmm-on-gcp-onboarding % vi 0-bootstrap/terraform.example.tfvars
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git diff
diff --git a/0-bootstrap/terraform.example.tfvars b/0-bootstrap/terraform.example.tfvars
index 7112195..2bc82a1 100644
--- a/0-bootstrap/terraform.example.tfvars
+++ b/0-bootstrap/terraform.example.tfvars
@@ -1,5 +1,5 @@
 /**
- * Copyright 2023 Google LLC
+ * Copyright 2024 Google LLC
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git add 0-bootstrap/terraform.example.tfvars 
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git status
On branch gh399-ado
Your branch is up to date with 'origin/gh399-ado'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
    modified:   0-bootstrap/terraform.example.tfvars

michaelobrien@mbp7 pbmm-on-gcp-onboarding % git commit -m "#399 - force tf.ex.tfvars sync2"
[gh399-ado 2a81519] #399 - force tf.ex.tfvars sync2
 1 file changed, 1 insertion(+), 1 deletion(-)
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git push origin gh399-ado                      
Enumerating objects: 7, done.
Counting objects: 100% (7/7), done.
Delta compression using up to 10 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 376 bytes | 376.00 KiB/s, done.
Total 4 (delta 3), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
remote: 
remote: GitHub found 4 vulnerabilities on GoogleCloudPlatform/pbmm-on-gcp-onboarding's default branch (2 critical, 2 moderate). To find out more, visit:
remote:      https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/security/dependabot
remote: 
To github.com:GoogleCloudPlatform/pbmm-on-gcp-onboarding.git
   c7e50e9..2a81519  gh399-ado -> gh399-ado

michaelobrien@mbp7 pbmm-on-gcp-onboarding % git remote -v
origin  git@github.com:GoogleCloudPlatform/pbmm-on-gcp-onboarding.git (fetch)
origin  git@github.com:GoogleCloudPlatform/pbmm-on-gcp-onboarding.git (push)
upstream    https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git (fetch)
upstream    https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git (push)

issue is origin and upstream are now the same
Screenshot 2024-05-05 at 13 01 04

Fixed upstream ADO merge - verify repo and branch

michaelobrien@mbp7 pbmm-on-gcp-onboarding % git remote -v
origin  git@github.com:GoogleCloudPlatform/pbmm-on-gcp-onboarding.git (fetch)
origin  git@github.com:GoogleCloudPlatform/pbmm-on-gcp-onboarding.git (push)
upstream    https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git (fetch)
upstream    https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git (push)
michaelobrien@mbp7 pbmm-on-gcp-onboarding % cd ..
michaelobrien@mbp7 ado_olxyz % rm -rf pbmm-on-gcp-onboarding 
michaelobrien@mbp7 ado_olxyz % git clone https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding
Cloning into 'pbmm-on-gcp-onboarding'...
remote: Azure Repos
remote: Found 6527 objects to send. (34 ms)
Receiving objects: 100% (6527/6527), 31.94 MiB | 28.94 MiB/s, done.
Resolving deltas: 100% (3971/3971), done.
michaelobrien@mbp7 ado_olxyz % cd pbmm-on-gcp-onboarding 
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git remote -v                                                                                                 
origin  https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding (fetch)
origin  https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding (push)
michaelobrien@mbp7 pbmm-on-gcp-onboarding % vi 0-bootstrap/terraform.example.tfvars                                                                       
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git checkout main
branch 'main' set up to track 'origin/main'.
Switched to a new branch 'main'
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git remote add upstream https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git                     
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git remote -v                                                                            
origin  https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding (fetch)
origin  https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding (push)
upstream    https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git (fetch)
upstream    https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git (push)
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git fetch upstream                                                                                            
remote: Enumerating objects: 53, done.
remote: Counting objects: 100% (53/53), done.
remote: Compressing objects: 100% (22/22), done.
remote: Total 53 (delta 34), reused 49 (delta 31), pack-reused 0
Unpacking objects: 100% (53/53), 25.34 KiB | 741.00 KiB/s, done.
From https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding
 * [new branch]      20240504-main-cb-ready                                         -> upstream/20240504-main-cb-ready
 * [new branch]      243-tef-retrofit                                               -> upstream/243-tef-retrofit
 * [new branch]      318-log-sink-alerting                                          -> upstream/318-log-sink-alerting
 * [new branch]      332-dev-prov-client-v20230917                                  -> upstream/332-dev-prov-client-v20230917
 * [new branch]      341-labels-off-332                                             -> upstream/341-labels-off-332
 * [new branch]      341-tags-off-332                                               -> upstream/341-tags-off-332
 * [new branch]      345-landing-zone-fortigate-light                               -> upstream/345-landing-zone-fortigate-light
 * [new branch]      351-landing-zone-fortigate-cloud-setup                         -> upstream/351-landing-zone-fortigate-cloud-setup
 * [new branch]      craigenator                                                    -> upstream/craigenator
 * [new branch]      craigenator-network                                            -> upstream/craigenator-network
 * [new branch]      dan-fix-version-upgrade                                        -> upstream/dan-fix-version-upgrade
 * [new branch]      dependabot/go_modules/helpers/foundation-deployer/github.com/hashicorp/go-getter-1.7.4 -> upstream/dependabot/go_modules/helpers/foundation-deployer/github.com/hashicorp/go-getter-1.7.4
 * [new branch]      dependabot/go_modules/helpers/foundation-deployer/golang.org/x/net-0.23.0 -> upstream/dependabot/go_modules/helpers/foundation-deployer/golang.org/x/net-0.23.0
 * [new branch]      dependabot/go_modules/test/integration/github.com/hashicorp/go-getter-1.7.4 -> upstream/dependabot/go_modules/test/integration/github.com/hashicorp/go-getter-1.7.4
 * [new branch]      dependabot/go_modules/test/integration/golang.org/x/net-0.23.0 -> upstream/dependabot/go_modules/test/integration/golang.org/x/net-0.23.0
 * [new branch]      example                                                        -> upstream/example
 * [new branch]      fmichaelobrien                                                 -> upstream/fmichaelobrien
 * [new branch]      fmichaelobrien-example-nonprod                                 -> upstream/fmichaelobrien-example-nonprod
 * [new branch]      fmichaelobrien-example-prod                                    -> upstream/fmichaelobrien-example-prod
 * [new branch]      fmichaelobrien-pr-113                                          -> upstream/fmichaelobrien-pr-113
 * [new branch]      fortigate-yopps                                                -> upstream/fortigate-yopps
 * [new branch]      gh357-tef-v4-fork                                              -> upstream/gh357-tef-v4-fork
 * [new branch]      gh360-day0-deploy-example                                      -> upstream/gh360-day0-deploy-example
 * [new branch]      gh362-bootstrap-prereq                                         -> upstream/gh362-bootstrap-prereq
 * [new branch]      gh377-architecture                                             -> upstream/gh377-architecture
 * [new branch]      gh387-upstream-sync                                            -> upstream/gh387-upstream-sync
 * [new branch]      gh399-ado                                                      -> upstream/gh399-ado
 * [new branch]      hotfix/cloudbuildjob                                           -> upstream/hotfix/cloudbuildjob
 * [new branch]      hotfix/guardrailsiam                                           -> upstream/hotfix/guardrailsiam
 * [new branch]      main                                                           -> upstream/main
 * [new branch]      pr-mro-bootstrap-localization                                  -> upstream/pr-mro-bootstrap-localization
 * [new branch]      pr-mro-bootstrap-run-terraform-local-041524                    -> upstream/pr-mro-bootstrap-run-terraform-local-041524
 * [new branch]      pr-mro-tef-fix-bucket-name-length-041524                       -> upstream/pr-mro-tef-fix-bucket-name-length-041524
 * [new branch]      pr-mro-tef-multi-instance-041524                               -> upstream/pr-mro-tef-multi-instance-041524
 * [new branch]      tm-lz-org                                                      -> upstream/tm-lz-org
 * [new branch]      v020                                                           -> upstream/v020
 * [new branch]      v20240407                                                      -> upstream/v20240407
michaelobrien@mbp7 pbmm-on-gcp-onboarding % 
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git merge upstream/main main                                                                                  
Already up to date.

michaelobrien@mbp7 pbmm-on-gcp-onboarding % git remote -v                
origin  https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding (fetch)
origin  https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding (push)
upstream    https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git (fetch)
upstream    https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git (push)
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git status
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git checkout gh399-ado
hint: If you meant to check out a remote tracking branch on, e.g. 'origin',
hint: you can do so by fully qualifying the name with the --track option:
hint: 
hint:     git checkout --track origin/<name>
hint: 
hint: If you'd like to always have checkouts of an ambiguous <name> prefer
hint: one remote, e.g. the 'origin' remote, consider setting
hint: checkout.defaultRemote=origin in your config.
fatal: 'gh399-ado' matched multiple (2) remote tracking branches
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git checkout --track origin/gh399-ado
branch 'gh399-ado' set up to track 'origin/gh399-ado'.
Switched to a new branch 'gh399-ado'

michaelobrien@mbp7 pbmm-on-gcp-onboarding % vi 0-bootstrap/terraform.example.tfvars                                                  
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git merge upstream/gh399-ado gh399-ado                                                                        
Merge made by the 'ort' strategy.
 0-bootstrap/README-Azure-DevOps.md   | 115 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 0-bootstrap/README.md                |  19 +++++++++++++++++++
 0-bootstrap/terraform.example.tfvars |  18 +++++++++++++++++-
 0-bootstrap/variables.tf             |   7 +++++++
 0-bootstrap/versions.tf              |   7 +++++++
 5 files changed, 160 insertions(+), 6 deletions(-)

 michaelobrien@mbp7 pbmm-on-gcp-onboarding % git status
On branch gh399-ado
Your branch is ahead of 'origin/gh399-ado' by 14 commits.
  (use "git push" to publish your local commits)

nothing to commit, working tree clean
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git commit -m "#399 - github to ado upstream merge"
On branch gh399-ado
Your branch is ahead of 'origin/gh399-ado' by 14 commits.
  (use "git push" to publish your local commits)

nothing to commit, working tree clean
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git push origin gh399-ado                                                                                     
Enumerating objects: 74, done.
Counting objects: 100% (68/68), done.
Delta compression using up to 10 threads
Compressing objects: 100% (54/54), done.
Writing objects: 100% (54/54), 14.85 KiB | 14.85 MiB/s, done.
Total 54 (delta 40), reused 0 (delta 0), pack-reused 0
remote: Analyzing objects... (54/54) (10 ms)
remote: Validating commits... (14/14) done (1 ms)
remote: Storing packfile... done (89 ms)
remote: Storing index... done (63 ms)
To https://dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding
   bb6d4e0..bc6bc4c  gh399-ado -> gh399-ado
Screenshot 2024-05-05 at 13 13 52 Screenshot 2024-05-05 at 13 14 20
obriensystems commented 1 month ago

revisit 0-bootstrap

just after the step to cd ./envs/shared step 7 of github and 6.1.8 of ado

nothing to commit, working tree clean michaelobrien@mbp7 gcp-bootstrap % ls README.md michaelobrien@mbp7 gcp-bootstrap % git checkout -b production Switched to a new branch 'production' michaelobrien@mbp7 gcp-bootstrap % ls README.md michaelobrien@mbp7 gcp-bootstrap % git checkout -b plan
Switched to a new branch 'plan' michaelobrien@mbp7 gcp-bootstrap % git status On branch plan nothing to commit, working tree clean michaelobrien@mbp7 gcp-bootstrap % mkdir -p envs/shared michaelobrien@mbp7 gcp-bootstrap % cp -R ../pbmm-on-gcp-onboarding/0-bootstrap/ ./envs/shared michaelobrien@mbp7 gcp-bootstrap % cp -R ../pbmm-on-gcp-onboarding/policy-library/ ./policy-library michaelobrien@mbp7 gcp-bootstrap % mkdir -p .github/workflows michaelobrien@mbp7 gcp-bootstrap % cp ../pbmm-on-gcp-onboarding/build/github-tf-* ./.github/workflows/ michaelobrien@mbp7 gcp-bootstrap % cp ../pbmm-on-gcp-onboarding/build/tf-wrapper.sh . michaelobrien@mbp7 gcp-bootstrap % chmod 755 ./tf-wrapper.sh michaelobrien@mbp7 gcp-bootstrap % cd ./envs/shared michaelobrien@mbp7 shared % mv ./cb.tf ./cb.tf.example michaelobrien@mbp7 shared % mv ./ado.tf.example ./ado.tf


<img width="1310" alt="Screenshot 2024-05-06 at 14 02 02" src="https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/assets/24765473/34531f72-aa20-44bf-9ba6-e38203b3026a">
obriensystems commented 1 month ago

restart ADO work 20240527

michaelobrien@mbp7 _deploy_test_399_from_ado % cd pbmm-on-gcp-onboarding 
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git status
On branch gh399-ado
Your branch is up to date with 'origin/gh399-ado'.

nothing to commit, working tree clean
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git pull
fatal: Authentication failed for 'https://dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding/'
michaelobrien@mbp7 pbmm-on-gcp-onboarding % pwd        
/Users/michaelobrien/wse_github/GoogleCloudPlatform/olapp/_deploy_test_399_from_ado/pbmm-on-gcp-onboarding

see https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/wiki/DevOps#ado-ssh-key

ADO SSH key

# on mac
ssh-keygen -t rsa -b 4096 -C "mic..yz" 
michaelobrien@mbp7 pbmm-on-gcp-onboarding % cp ado_olxyz.* ~/keys 
michaelobrien@mbp7 pbmm-on-gcp-onboarding % cp ado_olxyz ~/keys       
michaelobrien@mbp7 pbmm-on-gcp-onboarding % chmod 400 ~/keys/ado_olxyz
michaelobrien@mbp7 pbmm-on-gcp-onboarding % ssh-add ~/keys/ado_olxyz
Identity added: /Users/michaelobrien/keys/ado_olxyz (michael@obrienlabs.xyz)
michaelobrien@mbp7 pbmm-on-gcp-onboarding % cat ~/keys/ado_olxyz.pub 

paste to https://dev.azure.com/obrienlabsxyz/_usersSettings/keys
Screenshot 2024-05-27 at 12 14 08

use token

nel...koq
michaelobrien@mbp7 1 % git clone https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/gcp-bootstrap gcp-bootstrap
Cloning into 'gcp-bootstrap'...
Password for 'https://obrienlabsxyz@dev.azure.com': 
remote: Azure Repos
remote: Found 4 objects to send. (56 ms)
Unpacking objects: 100% (4/4), 899 bytes | 299.00 KiB/s, done.
michaelobrien@mbp7 1 % pwd
/Users/michaelobrien/wse_github/GoogleCloudPlatform/olapp/_deploy_test_399_from_ado/1

clone repo
michaelobrien@mbp7 _deploy_test_399_from_ado % rm -rf 1 
michaelobrien@mbp7 _deploy_test_399_from_ado % git clone https://obrienlabsxyz@dev.azure.com/obrienlabsxyz/pbmm-on-gcp-onboarding/_git/pbmm-on-gcp-onboarding       
Cloning into 'pbmm-on-gcp-onboarding'...
remote: Azure Repos
remote: Found 6581 objects to send. (28 ms)
Receiving objects: 100% (6581/6581), 31.96 MiB | 18.00 MiB/s, done.
Resolving deltas: 100% (4011/4011), done.
michaelobrien@mbp7 _deploy_test_399_from_ado % ls
gcp-bootstrap       pbmm-on-gcp-onboarding  terraform
michaelobrien@mbp7 _deploy_test_399_from_ado % cd pbmm-on-gcp-onboarding 
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git status
On branch 243-tef-retrofit
Your branch is up to date with 'origin/243-tef-retrofit'.

nothing to commit, working tree clean
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git checkout main
branch 'main' set up to track 'origin/main'.
Switched to a new branch 'main'
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git status
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
michaelobrien@mbp7 pbmm-on-gcp-onboarding % git checkout gh399-ado
branch 'gh399-ado' set up to track 'origin/gh399-ado'.
Switched to a new branch 'gh399-ado'

setup new build from branch with existing yml

Screenshot 2024-05-27 at 13 15 25 Screenshot 2024-05-27 at 13 15 51 Screenshot 2024-05-27 at 13 16 14 Screenshot 2024-05-27 at 13 16 31

working with

# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger:
- gh399-ado

pool:
  vmImage: ubuntu-latest

steps:
- script: echo Hello, world!
  displayName: 'Run a one-line script'

- script: |
    docker run obrienlabs/terraform-example-foundation-ado:0.0.2 --version
Screenshot 2024-05-27 at 13 26 51
fmichaelobrien commented 1 month ago

Billing setup after 30 day

https://learn.microsoft.com/en-us/azure/devops/organizations/billing/set-up-billing-for-your-organization-vs?view=azure-devops

Screenshot 2024-05-28 at 12 47 43 PM Screenshot 2024-05-28 at 12 42 30 PM Screenshot 2024-05-28 at 12 43 33 PM

Need to add test plans to users

https://learn.microsoft.com/en-us/azure/devops/organizations/billing/buy-basic-access-add-users?view=azure-devops

Screenshot 2024-05-28 at 12 45 32 PM Screenshot 2024-05-28 at 12 45 52 PM

still working

Screenshot 2024-05-28 at 12 47 19 PM