search

canada-ca / cloud-guardrails-gcp

Recommended configuration guidance for Google Cloud Platform / Conseils de configuration recommandés pour Platforme infonuagique de Google
Other
22 stars 14 forks source link

Validation verification #10

Open fmichaelobrien opened 2 years ago

fmichaelobrien commented 2 years ago

Doing a run on a recent guardrails install following https://github.com/canada-ca/cloud-guardrails-gcp/blob/main/guardrails-validation/README.md from https://github.com/canada-ca/accelerators_accelerateurs-gcp/issues/40 https://github.com/canada-ca/accelerators_accelerateurs-gcp/tree/main/deployment-templates/Terraform/guardrails

previous on a-r@nuage-cloud.info
cd accelerators_accelerateurs-gcp/
   25  gcloud services enable cloudasset.googleapis.com
   26  export MY_BUCKET_NAME=sccninfo-guardrails-validator
   27  gsutil mb gs://$MY_BUCKET_NAME
   28  gcloud asset export --output-path=gs://$MY_BUCKET_NAME/resource_inventory.json --content-type=resource --project=guardrails-eaba
   29  gcloud config set project guardrails-eaba
   30  gcloud services enable cloudasset.googleapis.com
   31  export MY_BUCKET_NAME=sccninfo-guardrails-validator-gr
   32  gsutil mb gs://$MY_BUCKET_NAME
   33  gcloud config set project sscncinfo-seed-project
   34  gcloud services enable cloudasset.googleapis.com
   35  gcloud config set project sscncinfo-seed-project-seed
   36  gcloud config set project sscncinfo-seed-project
   37  gcloud services enable cloudasset.googleapis.com
   38  export MY_BUCKET_NAME=sccninfo-guardrails-validator-seed
   39  gsutil mb gs://$MY_BUCKET_NAME
   40  history
   41  gcloud config set project guardrails-eaba
   42  gcloud config set project accelerator-nc-info

   Continue
admin_root@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp (accelerator-nc-info)$ gcloud config set project guardrails-eaba
Updated property [core/project].
   admin_root@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp (guardrails-eaba)$ gcloud services enable cloudasset.googleapis.com
admin_root@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp (guardrails-eaba)$ export MY_BUCKET_NAME=sccninfo-guardrails-validator-seed
admin_root@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp (guardrails-eaba)$ gsutil mb gs://$MY_BUCKET_NAME
Creating gs://sccninfo-guardrails-validator-seed/...
PreconditionException: 412 'us' violates constraint 'constraints/gcp.resourceLocations'

   admin_root@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp (guardrails-eaba)$ gcloud compute project-info describe --project guardrails-eaba | grep google-compute-default-region
admin_root@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp (guardrails-eaba)$ gcloud compute project-info add-metadata --metadata google-compute-default-region=northamerica-northeast1,google-compute-default-zone=northamerica-northeast1a
Updated [https://www.googleapis.com/compute/v1/projects/guardrails-eaba].
admin_root@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp (guardrails-eaba)$ gcloud compute project-info describe --project guardrails-eaba | grep google-compute-default-region
  - key: google-compute-default-region
admin_root@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp (guardrails-eaba)$ gsutil mb -l northamerica-northeast1 gs://$MY_BUCKET_NAME
Creating gs://sccninfo-guardrails-validator-seed/...
admin_root@cloudshell:~/cloudshell_open/accelerators_accelerateurs-gcp (guardrails-eaba)$ gsutil ls
gs://bkt-guardrails-eaba-org-logs-5ufo/
gs://sccninfo-guardrails-validator-seed/
gs://sscncinfo-guardrails-assets/
obriensystems commented 2 years ago

Workaround - turn off regional restriction on the project - or delete it on the parent and the rerun a terraform apply to get it back after

admin_@cloudshell:~/cloudshell_open (gr-bootstrap-ggz)$ gcloud services enable cloudasset.googleapis.com
Operation "operations/acat.p2-502392433631-09e81fe7-570c-44a3-8345-9852d82fd884" finished successfully.
admin_@cloudshell:~/cloudshell_open (gr-bootstrap-ggz)$ export MY_BUCKET_NAME=validation-ggz
admin_@cloudshell:~/cloudshell_open (gr-bootstrap-ggz)$ gsutil mb gs://$MY_BUCKET_NAME
Creating gs://validation-ggz/...
admin_@cloudshell:~/cloudshell_open (gr-bootstrap-ggz)$ gcloud asset export --output-path=gs://$MY_BUCKET_NAME/resource_inventory.json --content-type=resource --project=gr-bootstrap-ggz
Export in progress for root asset [projects/gr-bootstrap-ggz].
Use [gcloud asset operations describe projects/502392433631/operations/ExportAssets/RESOURCE/c6cfd41c3c7720348b468221cf6c688e] to check the status of the operation.
admin_@cloudshell:~/cloudshell_open (gr-bootstrap-ggz)

admin_@cloudshell:~/cloudshell_open (gr-bootstrap-ggz)$ git clone https://github.com/canada-ca/cloud-guardrails-gcp.git

admin_@cloudshell:~/cloudshell_open (gr-bootstrap-ggz)$ ls
accelerators_accelerateurs-gcp  cloud-guardrails-gcp
admin_@cloudshell:~/cloudshell_open (gr-bootstrap-ggz)$ gsutil cp gs://$MY_BUCKET_NAME/resource_inventory.json ./assets
Copying gs://validation-ggz/resource_inventory.json...
/ [1 files][ 16.8 KiB/ 16.8 KiB]
Operation completed over 1 objects/16.8 KiB.
admin_@cloudshell:~/cloudshell_open (gr-bootstrap-ggz)

dmin_@cloudshell:~/cloudshell_open (gr-bootstrap-ggz)$ cd
accelerators_accelerateurs-gcp/ cloud-guardrails-gcp/
admin_@cloudshell:~/cloudshell_open (gr-bootstrap-ggz)$ cd cloud-guardrails-gcp/
.git/                  guardrails/            guardrails-validation/
admin_@cloudshell:~/cloudshell_open (gr-bootstrap-ggz)$ cd cloud-guardrails-gcp/guardrails-validation/
admin_@cloudshell:~/cloudshell_open/cloud-guardrails-gcp/guardrails-validation (gr-bootstrap-ggz)$ ls
assets  cloudbuild.yaml  Dockerfile  install.sh  policies  README.md  run-all.sh  run.sh  tests.sh
admin_@cloudshell:~/cloudshell_open/cloud-guardrails-gcp/guardrails-validation (gr-bootstrap-ggz)$

min_@cloudshell:~/cloudshell_open/cloud-guardrails-gcp/guardrails-validation (gr-bootstrap-ggz)$ ./install.sh
--2022-11-20 15:39:04--  https://github.com/open-policy-agent/conftest/releases/download/v0.32.1/conftest_0.32.1_Linux_x86_64.tar.gz
Resolving github.com (github.com)... 140.82.113.3
Connecting to github.com (github.com)|140.82.113.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/178249461/a9c964a8-a471-41f8-aed7-86bca64ad3f8?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20221120%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20221120T153904Z&X-Amz-Expires=300&X-Amz-Signature=7b360ba6a1ab670e8c8957132cfcfda8d28cb797571ec78759636b3b6e402da8&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=178249461&response-content-disposition=attachment%3B%20filename%3Dconftest_0.32.1_Linux_x86_64.tar.gz&response-content-type=application%2Foctet-stream [following]
--2022-11-20 15:39:04--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/178249461/a9c964a8-a471-41f8-aed7-86bca64ad3f8?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20221120%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20221120T153904Z&X-Amz-Expires=300&X-Amz-Signature=7b360ba6a1ab670e8c8957132cfcfda8d28cb797571ec78759636b3b6e402da8&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=178249461&response-content-disposition=attachment%3B%20filename%3Dconftest_0.32.1_Linux_x86_64.tar.gz&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.110.133, 185.199.111.133, 185.199.108.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.110.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12264521 (12M) [application/octet-stream]
Saving to: ‘conftest_0.32.1_Linux_x86_64.tar.gz’

conftest_0.32.1_Linux_x86_64.tar.gz      100%[=================================================================================>]  11.70M  44.0MB/s    in 0.3s

2022-11-20 15:39:04 (44.0 MB/s) - ‘conftest_0.32.1_Linux_x86_64.tar.gz’ saved [12264521/12264521]

LICENSE
README.md
conftest
admin_@cloudshell:~/cloudshell_open/cloud-guardrails-gcp/guardrails-validation (gr-bootstrap-ggz)$

admin_@cloudshell:~/cloudshell_open/cloud-guardrails-gcp/guardrails-validation (gr-bootstrap-ggz)$ conftest --version
Conftest: 0.32.1
OPA: 0.40.0

run n/a due to missing json in the assets dir
dmin_@cloudshell:~/cloudshell_open/cloud-guardrails-gcp/guardrails-validation (gr-bootstrap-ggz)$ ./run.sh
Checking ./assets/*.json
cat: './assets/*.json': No such file or directory
admin_@cloudshell:~/cloudshell_open/cloud-guardrails-gcp/guardrails-validation (gr-bootstrap-ggz)$ ls ../
guardrails  guardrails-validation  LICENSE  README.md
admin_@cloudshell:~/cloudshell_open/cloud-guardrails-gcp/guardrails-validation (gr-bootstrap-ggz)$ ls
assets  cloudbuild.yaml  Dockerfile  install.sh  policies  README.md  report.txt  run-all.sh  run.sh  tests.sh
admin_@cloudshell:~/cloudshell_open/cloud-guardrails-gcp/guardrails-validation (gr-bootstrap-ggz)$ ls assets/
admin_@cloudshell:~/cloudshell_open/cloud-guardrails-gcp/guardrails-validation (gr-bootstrap-ggz)$

run-all also requires a rename of"my-unique-bucket-name" as well as addition storage admin role

admin_@cloudshell:~/cloudshell_open/cloud-guardrails-gcp/guardrails-validation (gr-bootstrap-ggz)$ ./run-all.sh
Your active configuration is: [cloudshell-22055]
Creating gs://my-unique-bucket-name/...
ServiceException: 409 A Cloud Storage bucket named 'my-unique-bucket-name' already exists. Try another name. Bucket names must be globally unique across all Google Cloud projects, including those outside of your organization.
ERROR: (gcloud.asset.export) code: 403
message: The billing account for the owning project is disabled in state closed
status: PERMISSION_DENIED
AccessDeniedException: 403 admin@guardrails.gcp.zone does not have storage.objects.list access to the Google Cloud Storage bucket. Permission 'storage.objects.list' denied on resource (or it may not exist).
cat: './assets/*.json': No such file or directory