fmichaelobrien commented 1 year ago

202401 update

https://cloud.google.com/architecture/serverless-blueprint
HCLS specific

Use Cases

the uc/customer-app drives the lz architecture subset. In this case we will start with 1 or more serverless use cases around profile 3.
docai, biometric telemetry, GPS tracking, event stream, live video ML/AI intelligence
see #220 for DocAI use case - https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/tree/canary/solutions/document-processing/gcloud
See LZ V1 traffic generation app https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/main/docs/google-cloud-landingzone-traffic-generation.md
See earlier V1 architecture discussions https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/blob/dev/solutions/landing-zone/architecture.md
See V2 architecture diagrams at https://docs.google.com/presentation/d/1Ztqn7G2rWFlssVwC7r5Xl-WFSFCbVo6MoHJffrXScmw/edit#slide=id.g25262d569c1_0_198
See GCP network-patterns share at https://docs.google.com/presentation/d/13sjT2tJ4yLIYGRREE3wBrylB1OvcEMpKdquVuJB_nX4/edit?resourcekey=0-N3DruQaiutFvZ98HTT7-vQ#slide=id.g1154b3b950f_2_3458
see Guardrails V1 https://github.com/canada-ca/cloud-guardrails/blob/master/EN/00_Applicable-Scope.md#applicability-of-guardrails-to-cloud-usage-profiles
see Guardrails V2 https://github.com/CloudLandingZone/cloud-guardrails-v2

Serverless Reference Architecture

251522916-5603ef02-e51e-47b6-a203-e2df4ecab0fd

Requirements: LZ workload Canary for Profile 3

P3 specific - we run a selected subset of all packages (armor yes, IDS/NGFW perimeter and extensive hub/spoke - no) for example
a VPC Service Control egress perimeter is involved
- 2+ microservices/containers - to exercise ingress/egress tracking
VPC Serverless Connector
No NGFW for east-west traffic
Armor attached to an LB (off the GFE)
SWP - Secure Web Proxy
External Cloud Load Balancer - https://cloud.google.com/load-balancing/docs/https
Artifact Registry and cloud build pipelines will work with the vpc-sc
hierarchical firewall policies at the 5 org/folder/vpc/region levels
the ALB in cloud run is not the Global LB
cloud run will use either a SQL proxy or vpc connector for private IP based cloud SQL connections
Ideally Guardrails V2 over V1
https://github.com/canada-ca/cloud-guardrails-v2/tree/main/EN
https://github.com/canada-ca/cloud-guardrails/blob/master/EN/00_Applicable-Scope.md#applicability-of-guardrails-to-cloud-usage-profiles

Design

Reference

Implementation

add to https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/tree/main/solutions/experimentation

Package Coverage Required

https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/tree/main/solutions/core-landing-zone
https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/tree/main/solutions/client-landing-zone or
https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/tree/main/solutions/experimentation/client-landing-zone
https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/tree/main/solutions/experimentation/client-project
Package Coverage Optional
Design Issues
watch CR https://github.com/GoogleCloudPlatform/k8s-config-connector/issues/832
lightweight JS frontend - minimal jquery/ajax - either encapsulated in container 1 or separate cloud functions knative app
container 1 : backend spring boot REST api (may contain javascript frontend as well)
container 2: backend Relational or key/value DB - may use Cloud SQL or Spanner

Template Introduction • Deliverables • MVP - Immediate Minimum Viable Product • Quickstart • Artifacts • Requirements • Features • Analysis • API • Architecture • Use Cases • Design • Design Issues • DevOps • Deployment • Testing • Security • Monitoring • Releases • Documentation • Development Log • Keywords • References • Links

reference as well https://github.com/ssc-spc-ccoe-cei/gcp-tier34-template

obriensystems commented 1 year ago

Implementation

review https://cloud.google.com/architecture/serverless-blueprint

fmichaelobrien commented 11 months ago

https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/blob/dev/solutions/landing-zone/architecture.md#di-25-continuous-delivery-pipelines ref: #432 ref: #430 ref: #418 ref: #339 ref: #344 ref: #192 ref: #409

fmichaelobrien commented 10 months ago

Verify VPC-SC across 2 service projects in a shared VPC

Goal/Requirements

prevent exfiltration of data from a cloud storage bucket or bigquery instance in two of the service projects hosting both of these services

Testing

source-svc can see target-svc selected services and vice versa
outside-svc - cannot see anything
outside-host (separate host project and VPC) - cannot see anything
follow slide 31 of the network-patterns deck https://docs.google.com/presentation/d/13sjT2tJ4yLIYGRREE3wBrylB1OvcEMpKdquVuJB_nX4/edit?resourcekey=0-N3DruQaiutFvZ98HTT7-vQ#slide=id.g1154b3b950f_2_3767
https://cloud.google.com/vpc-service-controls/docs/set-up-service-perimeter

VPC Service control zone must include both service projects Inventory

canary-host host project
canary-source-svc project
canary-target-svc project
vm-docker-source
vm-docker-target

enable compute api on all 3 projects In setup shared VPC - enable host on canary-host-ol -but first make a custom VPC and delete the default VPC

gcloud compute networks create dev --project=canary-host-ol --subnet-mode=custom --mtu=1460 --bgp-routing-mode=regional && gcloud compute networks subnets create dev-sn --project=canary-host-ol --range=10.10.0.0/24 --stack-type=IPV4_ONLY --network=dev --region=northamerica-northeast1 --enable-private-ip-google-access && gcloud compute firewall-rules create dev-allow-custom --project=canary-host-ol --network=projects/canary-host-ol/global/networks/dev --description=Allows\ connection\ from\ any\ source\ to\ any\ instance\ on\ the\ network\ using\ custom\ protocols. --direction=INGRESS --priority=65534 --source-ranges=10.10.0.0/24 --action=ALLOW --rules=all && gcloud compute firewall-rules create dev-allow-icmp --project=canary-host-ol --network=projects/canary-host-ol/global/networks/dev --description=Allows\ ICMP\ connections\ from\ any\ source\ to\ any\ instance\ on\ the\ network. --direction=INGRESS --priority=65534 --source-ranges=0.0.0.0/0 --action=ALLOW --rules=icmp && gcloud compute firewall-rules create dev-allow-rdp --project=canary-host-ol --network=projects/canary-host-ol/global/networks/dev --description=Allows\ RDP\ connections\ from\ any\ source\ to\ any\ instance\ on\ the\ network\ using\ port\ 3389. --direction=INGRESS --priority=65534 --source-ranges=0.0.0.0/0 --action=ALLOW --rules=tcp:3389 && gcloud compute firewall-rules create dev-allow-ssh --project=canary-host-ol --network=projects/canary-host-ol/global/networks/dev --description=Allows\ TCP\ connections\ from\ any\ source\ to\ any\ instance\ on\ the\ network\ using\ port\ 22. --direction=INGRESS --priority=65534 --source-ranges=0.0.0.0/0 --action=ALLOW --rules=tcp:22

share 2 svc projects

create vpc service control access policy at the org level

grant uses to the shared subnets

add projects to the perimeter

add resources to the perimeter to protect

Create VMs in subnets

gcloud compute instances create source-svc --project=canary-source-svc-ol --zone=northamerica-northeast1-a --machine-type=e2-small --network-interface=network-tier=PREMIUM,stack-type=IPV4_ONLY,subnet=projects/canary-host-ol/regions/northamerica-northeast1/subnetworks/dev-sn --maintenance-policy=MIGRATE --provisioning-model=STANDARD --service-account=531569623787-compute@developer.gserviceaccount.com --scopes=https://www.googleapis.com/auth/cloud-platform --create-disk=auto-delete=yes,boot=yes,device-name=source-svc,image=projects/debian-cloud/global/images/debian-11-bullseye-v20230814,mode=rw,size=10,type=projects/canary-source-svc-ol/zones/us-central1-a/diskTypes/pd-balanced --no-shielded-secure-boot --shielded-vtpm --shielded-integrity-monitoring --labels=goog-ec-src=vm_add-gcloud --reservation-affinity=any

gcloud compute instances create ext-svc --project=canary-ext-svc-ol --zone=northamerica-northeast1-a --machine-type=e2-small --network-interface=network-tier=PREMIUM,stack-type=IPV4_ONLY,subnet=projects/canary-ext-host-ol/regions/northamerica-northeast1/subnetworks/dev-sn --maintenance-policy=MIGRATE --provisioning-model=STANDARD --service-account=910458337292-compute@developer.gserviceaccount.com --scopes=https://www.googleapis.com/auth/cloud-platform --create-disk=auto-delete=yes,boot=yes,device-name=ext-svc,image=projects/debian-cloud/global/images/debian-11-bullseye-v20230814,mode=rw,size=10,type=projects/canary-ext-svc-ol/zones/us-central1-a/diskTypes/pd-balanced --no-shielded-secure-boot --shielded-vtpm --shielded-integrity-monitoring --labels=goog-ec-src=vm_add-gcloud --reservation-affinity=any

create a bucket with files to be accessed both inside and outside the perimeter in the source-svc project

Test access from ext-svc before/after perimeter is set

obriensystems commented 10 months ago

check VM access to bucket inside future perimeter

michael@source-svc:~$ gsutil ls gs://canary-source-svc-ol/
gs://canary-source-svc-ol/Easter Brunch Global Tables Takehome Notes 2023 Letter.pdf

michael@ext-svc:~$ gsutil ls
michael@ext-svc:~$ 
michael@ext-svc:~$ gsutil ls gs://canary-source-svc-ol/
AccessDeniedException: 403 910458337292-compute@developer.gserviceaccount.com does not have storage.objects.list access to the Google Cloud Storage bucket. Permission 'storage.objects.list' denied on resource (or it may not exist).

add the VM service account as reader

visible

michael@ext-svc:~$ gsutil ls gs://canary-source-svc-ol/
gs://canary-source-svc-ol/Easter Brunch Global Tables Takehome Notes 2023 Letter.pdf

set VPC service perimeter around the bucket specific to the source service project

add 2 projects to the perimeter

add cloud storage to the perimeter

checking - the reverse of what I expected - source is blocked, ext is not

michael@source-svc:~$ gsutil ls gs://canary-source-svc-ol/
gs://canary-source-svc-ol/Easter Brunch Global Tables Takehome Notes 2023 Letter.pdf

set projects in VPC-SC
michael@source-svc:~$ gsutil ls gs://canary-source-svc-ol/
AccessDeniedException: 403 Request is prohibited by organization's policy. vpcServiceControlsUniqueIdentifier: su8wcnumiqi2Urdl9NTTaymJAk_9AsbDaQef3wG8-wn_9zZwBuCSeQ

michael@ext-svc:~$ gsutil ls gs://canary-source-svc-ol/
gs://canary-source-svc-ol/Easter Brunch Global Tables Takehome Notes 2023 Letter.pdf

however turning on ext on and off in the perimeter sets the VPC SC flag - but I cannot get it reset

michael@ext-svc:~$ gsutil ls gs://canary-source-svc-ol/
AccessDeniedException: 403 Request is prohibited by organization's policy. vpcServiceControlsUniqueIdentifier: bPOAHVzoAq-3TnXxWL2yUnkOa8oLN9ub1sCEvKKqhbydgmzhWuYBV

test reduction only to target

delete perimeter but keep default policy - to reset

recreate perimeter

michael@source-svc:~$ gsutil ls gs://canary-source-svc-ol/
AccessDeniedException: 403 Request is prohibited by organization's policy. vpcServiceControlsUniqueIdentifier: i7PWPHxZaJjhRx5b_zD6ZoCZQ1TVRU8CC2-CwQU77xBceJ77wse7XA

michael@ext-svc:~$ gsutil ls gs://canary-source-svc-ol/
gs://canary-source-svc-ol/Easter Brunch Global Tables Takehome Notes 2023 Letter.pdf

VPC-SC: There are additional ingress and VPC owning project inclusion attributes to VPC Service Control - Perimeters in the policy such as per vpc/perimeter. Triaging VPC-SC config using a gs bucket in the notes starting at https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/issues/418#issuecomment-1688070495

reference https://cloud.google.com/vpc-service-controls/docs/overview#isolate

Missing the ingress rule and we also need the perimeter all the way around the host project as well - or else we use bridges

after the ingress rule we have the proper internal gsutil usage and blocked external gsutil usage

michael@ext-svc:~$ gsutil ls gs://canary-source-svc-ol/
AccessDeniedException: 403 Request is prohibited by organization's policy. vpcServiceControlsUniqueIdentifier: 5hL_K1kXZRP8W1icoLReQSeY3DlkWplevoH5Ga6YwvESsEC_OVdXBA

michael@source-svc:~$ gsutil ls gs://canary-source-svc-ol/
gs://canary-source-svc-ol/Easter Brunch Global Tables Takehome Notes 2023 Letter.pdf

GoogleCloudPlatform / pubsec-declarative-toolkit

Use Case: POC Serverless Canary Application (frontend/backend/persistence) as a Profile 3 LZ workload with PSC, PSA and VPC-SC #418

202401 update

Use Cases

Serverless Reference Architecture

Requirements: LZ workload Canary for Profile 3

Design

Implementation

Package Coverage Required

Package Coverage Optional

Design Issues

Implementation

Verify VPC-SC across 2 service projects in a shared VPC

Goal/Requirements

Testing

set VPC service perimeter around the bucket specific to the source service project