obriensystems
commented
1 year ago 20230710: update for partner interconnect
-
clean org with minimal terraform LZ - use the TOC version from https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/251#issuecomment-1629103711
-
add services project to host project
-
enable gce, gke
-
todo - services.tf update
michael@cloudshell:~$ gcloud config set project vpc-host-nonprod-hh015-gz357 michael@cloudshell:~ (vpc-host-nonprod-hh015-gz357)$ gcloud services enable container.googleapis.com michael@cloudshell:~ (vpc-host-nonprod-hh015-gz357)$ gcloud services enable compute.googleapis.com -
select non-prod host project
-
attach services project to host project
Prototype single BGP router and VLAN attachment from
with region additions
# https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_interconnect_attachment
resource "google_compute_interconnect_attachment" "on_prem1" {
name = "on-prem-attachment1"
edge_availability_domain = "AVAILABILITY_DOMAIN_1"
type = "PARTNER"
router = google_compute_router.router1.id
region = "northamerica-northeast2"
mtu = 1500
}
resource "google_compute_router" "router1" {
name = "router-1"
network = "vpc-nonprod-shared" #google_compute_network.network-ia.name
region = "northamerica-northeast2"
bgp {
asn = 16550
}
}
#resource "google_compute_network" "network-ia" {
# name = "network-ia"
# auto_create_subnetworks = false
#}
michael@cloudshell:~/tef_in_pbmm/vpchost-non-prod/pbmm-on-gcp-onboarding/2023_technical_onboarding_center/20-partner-interconnect (vpc-host-nonprod-hh015-gz357)$ terraform apply
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# google_compute_interconnect_attachment.on_prem1 will be created
+ resource "google_compute_interconnect_attachment" "on_prem1" {
+ admin_enabled = true
+ bandwidth = (known after apply)
+ cloud_router_ip_address = (known after apply)
+ creation_timestamp = (known after apply)
+ customer_router_ip_address = (known after apply)
+ edge_availability_domain = "AVAILABILITY_DOMAIN_1"
+ encryption = "NONE"
+ google_reference_id = (known after apply)
+ id = (known after apply)
+ mtu = "1500"
+ name = "on-prem-attachment1"
+ pairing_key = (known after apply)
+ partner_asn = (known after apply)
+ private_interconnect_info = (known after apply)
+ project = (known after apply)
+ region = (known after apply)
+ router = (known after apply)
+ self_link = (known after apply)
+ state = (known after apply)
+ type = "PARTNER"
+ vlan_tag8021q = (known after apply)
}
# google_compute_router.router1 will be created
+ resource "google_compute_router" "router1" {
+ creation_timestamp = (known after apply)
+ id = (known after apply)
+ name = "router-1"
+ network = "vpc-nonprod-shared"
+ project = (known after apply)
+ region = "northamerica-northeast2"
+ self_link = (known after apply)
+ bgp {
+ advertise_mode = "DEFAULT"
+ asn = 16550
+ keepalive_interval = 20
}
}
Plan: 2 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
google_compute_router.router1: Creating...
google_compute_router.router1: Still creating... [10s elapsed]
google_compute_router.router1: Still creating... [20s elapsed]
google_compute_router.router1: Creation complete after 21s [id=projects/vpc-host-nonprod-hh015-gz357/regions/northamerica-northeast2/routers/router-1]
google_compute_interconnect_attachment.on_prem1: Creating...
╷
│ Error: Cannot determine region: set in this resource, or set provider-level 'region' or 'zone'.
│
│ with google_compute_interconnect_attachment.on_prem1,
│ on main.tf line 23, in resource "google_compute_interconnect_attachment" "on_prem1":
│ 23: resource "google_compute_interconnect_attachment" "on_prem1" {
│
╵
michael@cloudshell:~/tef_in_pbmm/vpchost-non-prod/pbmm-on-gcp-onboarding/2023_technical_onboarding_center/20-partner-interconnect (vpc-host-nonprod-hh015-gz357)$ terraform plan
google_compute_router.router1: Refreshing state... [id=projects/vpc-host-nonprod-hh015-gz357/regions/northamerica-northeast2/routers/router-1]
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# google_compute_interconnect_attachment.on_prem1 will be created
+ resource "google_compute_interconnect_attachment" "on_prem1" {
+ admin_enabled = true
+ bandwidth = (known after apply)
+ cloud_router_ip_address = (known after apply)
+ creation_timestamp = (known after apply)
+ customer_router_ip_address = (known after apply)
+ edge_availability_domain = "AVAILABILITY_DOMAIN_1"
+ encryption = "NONE"
+ google_reference_id = (known after apply)
+ id = (known after apply)
+ mtu = "1500"
+ name = "on-prem-attachment1"
+ pairing_key = (known after apply)
+ partner_asn = (known after apply)
+ private_interconnect_info = (known after apply)
+ project = (known after apply)
+ region = "northamerica-northeast2"
+ router = "projects/vpc-host-nonprod-hh015-gz357/regions/northamerica-northeast2/routers/router-1"
+ self_link = (known after apply)
+ state = (known after apply)
+ type = "PARTNER"
+ vlan_tag8021q = (known after apply)
}
Plan: 1 to add, 0 to change, 0 to destroy.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.
michael@cloudshell:~/tef_in_pbmm/vpchost-non-prod/pbmm-on-gcp-onboarding/2023_technical_onboarding_center/20-partner-interconnect (vpc-host-nonprod-hh015-gz357)$
google_compute_interconnect_attachment.on_prem1: Creating...
google_compute_interconnect_attachment.on_prem1: Still creating... [10s elapsed]
google_compute_interconnect_attachment.on_prem1: Creation complete after 12s [id=projects/vpc-host-nonprod-hh015-gz357/regions/northamerica-northeast2/interconnectAttachments/on-prem-attachment1]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.Router and VLAN up - waiting on service provider as expected
google_compute_interconnect_attachment.on_prem1: Destroying... [id=projects/vpc-host-nonprod-hh015-gz357/regions/northamerica-northeast2/interconnectAttachments/on-prem-attachment1]
google_compute_interconnect_attachment.on_prem1: Still destroying... [id=projects/vpc-host-nonprod-hh015-gz357/r...connectAttachments/on-prem-attachment1, 10s elapsed]
google_compute_interconnect_attachment.on_prem1: Destruction complete after 11s
google_compute_router.router1: Destroying... [id=projects/vpc-host-nonprod-hh015-gz357/regions/northamerica-northeast2/routers/router-1]
google_compute_router.router1: Still destroying... [id=projects/vpc-host-nonprod-hh015-gz357/r...rthamerica-northeast2/routers/router-1, 10s elapsed]
google_compute_router.router1: Destruction complete after 11s
Destroy complete! Resources: 2 destroyed.add 2nd vlan attachment
oogle_compute_router.router1: Creating...
google_compute_router.router1: Still creating... [10s elapsed]
google_compute_router.router1: Creation complete after 12s [id=projects/vpc-host-nonprod-hh015-gz357/regions/northamerica-northeast2/routers/router-1]
google_compute_interconnect_attachment.on_prem2: Creating...
google_compute_interconnect_attachment.on_prem1: Creating...
google_compute_interconnect_attachment.on_prem2: Still creating... [10s elapsed]
google_compute_interconnect_attachment.on_prem1: Still creating... [10s elapsed]
google_compute_interconnect_attachment.on_prem1: Creation complete after 11s [id=projects/vpc-host-nonprod-hh015-gz357/regions/northamerica-northeast2/interconnectAttachments/on-prem-attachment1]
google_compute_interconnect_attachment.on_prem2: Creation complete after 11s [id=projects/vpc-host-nonprod-hh015-gz357/regions/northamerica-northeast2/interconnectAttachments/on-prem-attachment2]
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
Outputs:
fmichaelobrien
See private DNS work in #286 - as a result of transitive peering - add a 2nd interconnect set in non-prod in #289 Configure IaC in TF for partner interconnect overlay
https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/pull/258 https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/pull/259 https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/pull/260 https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/pull/261
shadow already prototyped cologix partner interconnect running in KCC LZ V2 https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/issues/298 Create Terraform version of Kubernetes Config Controller based work in 298
TODO
78
R1: Implement SCED/SC2G for GCP
R2: Use MacSEC over IPSec
R3: partner interconnect over direct connect
R4: ASNs Google: 16550
R5: Interconnect redundancy
R6: VPC Global Dynamic Routing and traffic shaping
Phase 1: brute force minimal connection test Phase 2: Architecture final review
Initial work item is to start an interconnect and get the SDN UID to tag traffic. Testing out https://cloud.google.com/network-connectivity/docs/interconnect/concepts/partner-overview#provisioning
Links
Slide 19, 18 were used a lot in presentations at IRCC and ISED (from 2019 but still valid) - for profile 5/6 https://wiki.gccollab.ca/images/7/75/GC_Cloud_Connection_Patterns.pdf review: https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/blueprints/networking/private-cloud-function-from-onprem review https://cloud.google.com/network-connectivity/docs/interconnect/concepts/ha-vpn-interconnect
slide 65 in progress in https://docs.google.com/presentation/d/13sjT2tJ4yLIYGRREE3wBrylB1OvcEMpKdquVuJB_nX4/edit?resourcekey=0-N3DruQaiutFvZ98HTT7-vQ#slide=id.g2223ba5ee2b_0_665
20230717
20230413
20230321
https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/blueprints/networking/psc-hybrid
Organization Policies
Architecture Notes