fortinet / fortigate-tutorial-gcp

6 stars 12 forks source link

Example fortigate-tutorial-gcp deployment #1

Closed fmichaelobrien closed 1 year ago

fmichaelobrien commented 1 year ago

Good afternoon team,. I will be reviewing this your implementation. Thank you for the excellent work. Keeping this issue open for initial discussion on config/deploy/use items. /Michael

obriensystems commented 1 year ago

following https://github.com/fortinet/fortigate-tutorial-gcp#how-to-deploy to https://github.com/fortinet/fortigate-tutorial-gcp/tree/main/terraform to https://github.com/fortinet/fortigate-tutorial-gcp/tree/main/terraform/day0

michael@cloudshell:~$ gcloud config set project fortigate-tutorial-gcp-lgz
Updated property [core/project].
michael@cloudshell:~ (fortigate-tutorial-gcp-lgz)$ cd github/
michael@cloudshell:~/github (fortigate-tutorial-gcp-lgz)$ git clone https://github.com/fortinet/fortigate-tutorial-gcp.git
Cloning into 'fortigate-tutorial-gcp'...
michael@cloudshell:~/github (fortigate-tutorial-gcp-lgz)$ cd fortigate-tutorial-gcp/
michael@cloudshell:~/github/fortigate-tutorial-gcp (fortigate-tutorial-gcp-lgz)$ chmod 755 service_account_create.sh
michael@cloudshell:~/github/fortigate-tutorial-gcp (fortigate-tutorial-gcp-lgz)$ ./service_account_create.sh

michael@cloudshell:~/github/fortigate-tutorial-gcp (fortigate-tutorial-gcp-lgz)$ ./service_account_create.sh
Your active configuration is: [cloudshell-10076]
Creating FortigateSdnReader role in project fortigate-tutorial-gcp-lgz...
WARNING: API is not enabled for permissions: [compute.zones.list, compute.instances.list, container.clusters.list, container.nodes.list, container.pods.list, container.services.list]. Please enable the corresponding APIs to use those permissions.

Created role [FortigateSdnReader].
etag: BwXrZN1m7_k=
includedPermissions:
- compute.instances.list
- compute.zones.list
- container.clusters.list
- container.nodes.list
- container.pods.list
- container.services.list
name: projects/fortigate-tutorial-gcp-lgz/roles/FortigateSdnReader
stage: ALPHA
title: FortiGate SDN Connector Role (read-only)
Creating new service account (FortiGate SDN Connector)...
Created service account [fortigatesdn-ro].
Granting fortigatesdn-ro service account access to project fortigate-tutorial-gcp-lgz...
Updated IAM policy for project [fortigate-tutorial-gcp-lgz].
bindings:
- members:
  - serviceAccount:fortigatesdn-ro@fortigate-tutorial-gcp-lgz.iam.gserviceaccount.com
  role: projects/fortigate-tutorial-gcp-lgz/roles/FortigateSdnReader
- members:
  - user:michael@landing.gcp.zone
  role: roles/owner
etag: BwXrZN2P0Qs=
version: 1
serviceAccount:fortigatesdn-ro@fortigate-tutorial-gcp-lgz.iam.gserviceaccount.com
Service account created succesfully

enabling services separately

michael@cloudshell:~/github/fortigate-tutorial-gcp (fortigate-tutorial-gcp-lgz)$ gcloud services enable compute.googleapis.com
michael@cloudshell:~/github/fortigate-tutorial-gcp (fortigate-tutorial-gcp-lgz)$ gcloud services enable container.googleapis.com

Use fortigate pay as you go - comment out 31-34 of https://github.com/fortinet/fortigate-tutorial-gcp/blob/main/terraform/day0/main.tf#L31

  license_files   = [
    "lic1.lic",
    "lic2.lic"
  ]

https://github.com/fortinet/fortigate-tutorial-gcp/blob/main/terraform/modules/fgcp-ha-ap-lb/main.tf#L5

data "google_compute_image" "fgt_image" {
  project         = "fortigcp-project-001"
  #family          = "fortigate-70-byol"
  family          = "fortigate-70-payg"

modify terraform.tfvars

GCP_PROJECT = "fortigate-tutorial-gcp-lgz"
GCE_REGION = "northamerica-northeast1"
prefix = "fgt-"

add to step 7 by cd into the terraform/day0 dir https://github.com/fortinet/fortigate-tutorial-gcp/tree/main/terraform/day0#how-to-deploy

michael@cloudshell:~/github/fortigate-tutorial-gcp (fortigate-tutorial-gcp-lgz)$ cd terraform/
michael@cloudshell:~/github/fortigate-tutorial-gcp/terraform (fortigate-tutorial-gcp-lgz)$ cd day0
michael@cloudshell:~/github/fortigate-tutorial-gcp/terraform/day0 (fortigate-tutorial-gcp-lgz)$ terraform init

Terraform has been successfully initialized!

Fix the license references to avoid the terraform errors around rendering the byol

michael@cloudshell:~/github/fortigate-tutorial-gcp/terraform/day0 (fortigate-tutorial-gcp-lgz)$ git diff
diff --git a/service_account_create.sh b/service_account_create.sh
old mode 100644
new mode 100755
diff --git a/terraform/day0/main.tf b/terraform/day0/main.tf
index 550804f..368a76e 100644
--- a/terraform/day0/main.tf
+++ b/terraform/day0/main.tf
@@ -28,10 +28,10 @@ module "fortigates" {
      "${var.prefix}sb-mgmt"
    ]

-  license_files   = [
-    "lic1.lic",
-    "lic2.lic"
-  ]
+#  license_files   = [
+#    "lic1.lic",
+#    "lic2.lic"
+#  ]

   # If creating sample VPC Networks in the same configuration - wait for them to be created!
   # Remove this explicit dependency if using your own pre-existing networks.
diff --git a/terraform/day0/terraform.tfvars b/terraform/day0/terraform.tfvars
index bd73283..31bf187 100644
--- a/terraform/day0/terraform.tfvars
+++ b/terraform/day0/terraform.tfvars
@@ -1,3 +1,6 @@
-GCP_PROJECT = "forti-emea-se"
-GCE_REGION = "europe-west3"
+#GCP_PROJECT = "forti-emea-se"
+#GCE_REGION = "europe-west3"
+#prefix = "fgt-"
+GCP_PROJECT = "fortigate-tutorial-gcp-lgz"
+GCE_REGION = "northamerica-northeast1"
 prefix = "fgt-"
diff --git a/terraform/modules/fgcp-ha-ap-lb/main.tf b/terraform/modules/fgcp-ha-ap-lb/main.tf
index c3db3e7..194d556 100644
--- a/terraform/modules/fgcp-ha-ap-lb/main.tf
+++ b/terraform/modules/fgcp-ha-ap-lb/main.tf
@@ -2,7 +2,8 @@
 # an image different than the newest 7.0 BYOL series.
 data "google_compute_image" "fgt_image" {
   project         = "fortigcp-project-001"
-  family          = "fortigate-70-byol"
+  #family          = "fortigate-70-byol"
+  family          = "fortigate-70-payg"
 }

 # Pull information about subnets we will connect to FortiGate instances. Subnets must
@@ -121,7 +122,7 @@ resource "google_compute_instance" "fgt-vm" {

   metadata = {
     user-data            = (count.index == 0 ? local.config_active : local.config_passive )
-    license              = fileexists(var.license_files[count.index]) ? file(var.license_files[count.index]) : null
+    #license              = fileexists(var.license_files[count.index]) ? file(var.license_files[count.index]) : null
   }

   network_interface {
diff --git a/terraform/modules/fgcp-ha-ap-lb/variables.tf b/terraform/modules/fgcp-ha-ap-lb/variables.tf
index 081eaa0..a1bc1c3 100644
--- a/terraform/modules/fgcp-ha-ap-lb/variables.tf
+++ b/terraform/modules/fgcp-ha-ap-lb/variables.tf
@@ -57,11 +57,11 @@ variable api_acl {
   description = "List of CIDRs allowed to connect to FortiGate API (must not be 0.0.0.0/0)"
 }

-variable license_files {
-  type        = list(string)
-  default     = ["",""]
-  description = "List of license (.lic) files to be applied for BYOL instances."
-}
+#variable license_files {
+#  type        = list(string)
+#  default     = ["",""]
+#  description = "List of license (.lic) files to be applied for BYOL instances."
+#}

 variable healthcheck_port {
   type        = number

terraform plan

michael@cloudshell:~/github/fortigate-tutorial-gcp/terraform/day0 (fortigate-tutorial-gcp-lgz)$ terraform plan -out day0.plan

Plan: 35 to add, 0 to change, 0 to destroy.
Saved the plan to: day0.plan

terraform apply

14:27

michael@cloudshell:~/github/fortigate-tutorial-gcp/terraform/day0 (fortigate-tutorial-gcp-lgz)$ terraform apply day0.plan
module.sample_networks.google_compute_network.vpcs[3]: Still creating... [10s elapsed]
module.sample_networks.google_compute_network.vpcs[1]: Creation complete after 12s [id=projects/fortigate-tutorial-gcp-lgz/global/networks/fgt--internal-vpc]
module.sample_networks.google_compute_network.vpcs[0]: Still creating... [20s elapsed]
module.sample_networks.google_compute_network.vpcs[2]: Still creating... [20s elapsed]
module.sample_networks.google_compute_network.vpcs[3]: Still creating... [20s elapsed]
module.sample_networks.google_compute_network.vpcs[0]: Creation complete after 22s [id=projects/fortigate-tutorial-gcp-lgz/global/networks/fgt--external-vpc]
module.sample_networks.google_compute_network.vpcs[3]: Creation complete after 22s [id=projects/fortigate-tutorial-gcp-lgz/global/networks/fgt--mgmt-vpc]
module.sample_networks.google_compute_network.vpcs[2]: Creation complete after 22s [id=projects/fortigate-tutorial-gcp-lgz/global/networks/fgt--hasy

Error: Invalid function argument
│
│   on ../modules/fgcp-ha-ap-lb/main.tf line 63, in locals:
│   63:     unicast_peer_netmask   = cidrnetmask(data.google_compute_subnetwork.subnets[2].ip_cidr_range)
│     ├────────────────
│     │ while calling cidrnetmask(prefix)
│     │ data.google_compute_subnetwork.subnets[2].ip_cidr_range is null
│
│ Invalid value for "prefix" parameter: argument must not be null.
╵
╷
│ Error: Invalid function argument
│
│   on ../modules/fgcp-ha-ap-lb/main.tf line 82, in locals:
│   82:     unicast_peer_netmask   = cidrnetmask(data.google_compute_subnetwork.subnets[2].ip_cidr_range)
│     ├────────────────
│     │ while calling cidrnetmask(prefix)
│     │ data.google_compute_subnetwork.subnets[2].ip_cidr_range is null
│
│ Invalid value for "prefix" parameter: argument must not be null.
╵
╷
│ Error: Missing required argument
│
│   with module.fortigates.google_compute_firewall.allow-mgmt,
│   on ../modules/fgcp-ha-ap-lb/main.tf line 205, in resource "google_compute_firewall" "allow-mgmt":
│  205:   network                = data.google_compute_subnetwork.subnets[3].network
│
│ The argument "network" is required, but no definition was found.
╵
╷
│ Error: Missing required argument
│
│   with module.fortigates.google_compute_firewall.allow-hasync,
│   on ../modules/fgcp-ha-ap-lb/main.tf line 216, in resource "google_compute_firewall" "allow-hasync":
│  216:   network                = data.google_compute_subnetwork.subnets[2].network
│
│ The argument "network" is required, but no definition was found.
╵
╷
│ Error: Missing required argument
│
│   with module.fortigates.google_compute_firewall.allow-port1,
│   on ../modules/fgcp-ha-ap-lb/main.tf line 227, in resource "google_compute_firewall" "allow-port1":
│  227:   network                = data.google_compute_subnetwork.subnets[0].network
│
│ The argument "network" is required, but no definition was found.
╵
╷
│ Error: Missing required argument
│
│   with module.fortigates.google_compute_firewall.allow-port2,
│   on ../modules/fgcp-ha-ap-lb/main.tf line 237, in resource "google_compute_firewall" "allow-port2":
│  237:   network                = data.google_compute_subnetwork.subnets[1].network
│
│ The argument "network" is required, but no definition was found.
╵
╷
│ Error: Missing required argument
│
│   with module.fortigates.google_compute_router.nat_router,
│   on ../modules/fgcp-ha-ap-lb/main.tf line 249, in resource "google_compute_router" "nat_router":
│  249:   network                = data.google_compute_subnetwork.subnets[0].network
│
│ The argument "network" is required, but no definition was found.
╵
fmichaelobrien commented 1 year ago

Have a feeling either it was my region or there the a reference issue between data.google_compute_subnetwork.subnets and the 4 vpc's created ok

michael@cloudshell:~/github/fortigate-tutorial-gcp/terraform/day0 (fortigate-tutorial-gcp-lgz)$ gcloud config set project fortigate-tutorial-gcp-lgz2
Updated property [core/project].
michael@cloudshell:~/github/fortigate-tutorial-gcp/terraform/day0 (fortigate-tutorial-gcp-lgz2)$ gcloud services enable compute.googleapis.com
Operation "operations/acf.p2-583553966478-9b43f99c-43b4-4f9a-9012-ab217cf396c8" finished successfully.
michael@cloudshell:~/github/fortigate-tutorial-gcp/terraform/day0 (fortigate-tutorial-gcp-lgz2)$ gcloud services enable container.googleapis.com
Operation "operations/acf.p2-583553966478-16f84d9b-4b6d-4a3f-8593-5cbe039c6e74" finished successfully.
michael@cloudshell:~/github/fortigate-tutorial-gcp/terraform/day0 (fortigate-tutorial-gcp-lgz2)$ cd ..
michael@cloudshell:~/github/fortigate-tutorial-gcp/terraform (fortigate-tutorial-gcp-lgz2)$ cd ..
michael@cloudshell:~/github/fortigate-tutorial-gcp (fortigate-tutorial-gcp-lgz2)$ ./
deployment-manager/        docs/                      gcloud/                    .git/                      service_account_create.sh  terraform/
michael@cloudshell:~/github/fortigate-tutorial-gcp (fortigate-tutorial-gcp-lgz2)$ ./service_account_create.sh
Your active configuration is: [cloudshell-17963]
Creating FortigateSdnReader role in project fortigate-tutorial-gcp-lgz2...
Created role [FortigateSdnReader].
etag: BwXraefoqUY=
includedPermissions:
- compute.instances.list
- compute.zones.list
- container.clusters.list
- container.nodes.list
- container.pods.list
- container.services.list
name: projects/fortigate-tutorial-gcp-lgz2/roles/FortigateSdnReader
stage: ALPHA
title: FortiGate SDN Connector Role (read-only)
Creating new service account (FortiGate SDN Connector)...
Created service account [fortigatesdn-ro].
Granting fortigatesdn-ro service account access to project fortigate-tutorial-gcp-lgz2...
Updated IAM policy for project [fortigate-tutorial-gcp-lgz2].
bindings:
- members:
  - serviceAccount:fortigatesdn-ro@fortigate-tutorial-gcp-lgz2.iam.gserviceaccount.com
  role: projects/fortigate-tutorial-gcp-lgz2/roles/FortigateSdnReader
- members:
  - serviceAccount:service-583553966478@compute-system.iam.gserviceaccount.com
  role: roles/compute.serviceAgent
- members:
  - serviceAccount:service-583553966478@container-engine-robot.iam.gserviceaccount.com
  role: roles/container.serviceAgent
- members:
  - serviceAccount:service-583553966478@containerregistry.iam.gserviceaccount.com
  role: roles/containerregistry.ServiceAgent
- members:
  - serviceAccount:583553966478-compute@developer.gserviceaccount.com
  - serviceAccount:583553966478@cloudservices.gserviceaccount.com
  role: roles/editor
- members:
  - user:michael@landing.gcp.zone
  role: roles/owner
- members:
  - serviceAccount:service-583553966478@gcp-sa-pubsub.iam.gserviceaccount.com
  role: roles/pubsub.serviceAgent
etag: BwXraegZKrM=
version: 1
serviceAccount:fortigatesdn-ro@fortigate-tutorial-gcp-lgz2.iam.gserviceaccount.com
Service account created succesfully

Plan: 34 to add, 0 to change, 22 to destroy.

Changes to Outputs:
  + default_password = (known after apply)
  ~ fgt-mgmt-eips    = [
      - "35.203.84.17",
      - "35.203.6.190",
      + (known after apply),
      + (known after apply),
    ]
  + fgt_umigs        = [
      + (known after apply),
      + (known after apply),
    ]
  ~ health_check     = "https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz/regions/northamerica-northeast1/healthChecks/fgthealthcheck-http8008-na-northeast1" -> (known after apply)
  + ilb              = (known after apply)
  + internal_subnet  = (known after apply)
  + internal_vpc     = (known after apply)
  ~ prefix           = "fgt-" -> "fgt2-"
  ~ project          = "fortigate-tutorial-gcp-lgz" -> "fortigate-tutorial-gcp-lgz2"
  ~ region           = "northamerica-northeast1" -> "europe-west3"
╷
│ Warning: Attribute Deprecated
│
│   with module.fortigates.random_string.api_key,
│   on ../modules/fgcp-ha-ap-lb/main.tf line 46, in resource "random_string" "api_key":
│   46:   number                 = true
│
│ **NOTE**: This is deprecated, use `numeric` instead.
│
│ (and one more similar warning elsewhere)
╵

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: day0.plan

To perform exactly these actions, run the following command to apply:
    terraform apply "day0.plan"

    1732

module.fortigates.google_compute_address.ext_priv[1]: Destruction complete after 12s
module.fortigates.google_compute_address.hasync_priv[0]: Destruction complete after 12s
module.fortigates.google_compute_address.ilb: Destruction complete after 12s
module.fortigates.google_compute_disk.logdisk[0]: Destruction complete after 12s
module.fortigates.google_compute_address.int_priv[0]: Still destroying... [id=projects/fortigate-tutorial-gcp-lgz/reg...1/addresses/fgtip1-trust-na-northeast1, 10s elapsed]
module.fortigates.google_compute_address.mgmt_pub[0]: Still destroying... [id=projects/fortigate-tutorial-gcp-lgz/reg...1/addresses/fgteip1-mgmt-na-northeast1, 10s elapsed]
module.fortigates.google_compute_address.ext_priv[0]: Still destroying... [id=projects/fortigate-tutorial-gcp-lgz/reg...addresses/fgtip1-untrust-na-northeast1, 10s elapsed]
module.fortigates.google_compute_address.mgmt_pub[1]: Still destroying... [id=projects/fortigate-tutorial-gcp-lgz/reg...1/addresses/fgteip2-mgmt-na-northeast1, 10s elapsed]
module.fortigates.google_compute_address.ext_priv[0]: Destruction complete after 12s
module.fortigates.google_compute_address.int_priv[0]: Destruction complete after 12s
module.fortigates.google_compute_address.mgmt_pub[0]: Destruction complete after 12s
module.fortigates.google_compute_address.mgmt_pub[1]: Destruction complete after 11s
module.sample_networks.google_compute_subnetwork.subnets[0]: Destroying... [id=projects/fortigate-tutorial-gcp-lgz/regions/northamerica-northeast1/subnetworks/fgt--external-sb]
module.sample_networks.google_compute_subnetwork.subnets[1]: Destroying... [id=projects/fortigate-tutorial-gcp-lgz/regions/northamerica-northeast1/subnetworks/fgt--internal-sb]
module.sample_networks.google_compute_subnetwork.subnets[3]: Destroying... [id=projects/fortigate-tutorial-gcp-lgz/regions/northamerica-northeast1/subnetworks/fgt--mgmt-sb]

module.fortigates.google_compute_address.ext_priv[0]: Creating...
module.fortigates.google_compute_disk.logdisk[0]: Creation complete after 2s [id=projects/fortigate-tutorial-gcp-lgz2/zones/europe-west3-a/disks/fgtdisk-logdisk1-euwest3-a]
module.fortigates.google_compute_address.ext_priv[0]: Creation complete after 2s [id=projects/fortigate-tutorial-gcp-lgz2/regions/europe-west3/addresses/fgtip1-untrust-euwest3]
module.fortigates.google_compute_address.ilb: Creation complete after 2s [id=projects/fortigate-tutorial-gcp-lgz2/regions/europe-west3/addresses/fgtip-ilb-trust-euwest3]
module.fortigates.google_compute_address.int_priv[1]: Creation complete after 2s [id=projects/fortigate-tutorial-gcp-lgz2/regions/europe-west3/addresses/fgtip2-trust-euwest3]

same - missing the created VPC's - switching to manually referencing them - or just gcloud
│ Error: Missing required argument
│
│   with module.fortigates.google_compute_router.nat_router,
│   on ../modules/fgcp-ha-ap-lb/main.tf line 249, in resource "google_compute_router" "nat_router":
│  249:   network                = data.google_compute_subnetwork.subnets[0].network
│
│ The argument "network" is required, but no definition was found.
╵
fmichaelobrien commented 1 year ago

gcloud https://github.com/fortinet/fortigate-tutorial-gcp/tree/main/gcloud

michael@cloudshell:~/github/fortigate-tutorial-gcp/terraform/day0 (fortigate-tutorial-gcp-lgz2)$ gcloud config set project fortigate-tutorial-gcp-lgz3
Updated property [core/project].
michael@cloudshell:~/github/fortigate-tutorial-gcp/terraform/day0 (fortigate-tutorial-gcp-lgz3)$ gcloud services enable container.googleapis.com

Operation "operations/acf.p2-672466585052-9a23ecca-43fe-4182-a304-8792c8e95bde" finished successfully.
michael@cloudshell:~/github/fortigate-tutorial-gcp/terraform/day0 (fortigate-tutorial-gcp-lgz3)$
michael@cloudshell:~/github/fortigate-tutorial-gcp/terraform/day0 (fortigate-tutorial-gcp-lgz3)$ gcloud services enable compute.googleapis.com
michael@cloudshell:~/github/fortigate-tutorial-gcp/terraform/day0 (fortigate-tutorial-gcp-lgz3)$ cd ..
michael@cloudshell:~/github/fortigate-tutorial-gcp/terraform (fortigate-tutorial-gcp-lgz3)$ cd ..
michael@cloudshell:~/github/fortigate-tutorial-gcp (fortigate-tutorial-gcp-lgz3)$ ./service_account_create.sh
michael@cloudshell:~/github/fortigate-tutorial-gcp/gcloud (fortigate-tutorial-gcp-lgz3)$ ./tutorial-create.sh
################################################################################
#
# I. VPCs and subnets
# --------------------
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/ext-vpc-global].
NAME: ext-vpc-global
SUBNET_MODE: CUSTOM
BGP_ROUTING_MODE: REGIONAL
IPV4_RANGE:
GATEWAY_IPV4:

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network ext-vpc-global --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network ext-vpc-global --allow tcp:22,tcp:3389,icmp

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/int-vpc-euwest1].
NAME: int-vpc-euwest1
SUBNET_MODE: CUSTOM
BGP_ROUTING_MODE: REGIONAL
IPV4_RANGE:
GATEWAY_IPV4:

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network int-vpc-euwest1 --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network int-vpc-euwest1 --allow tcp:22,tcp:3389,icmp

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/fgt-hasync-vpc].
NAME: fgt-hasync-vpc
SUBNET_MODE: CUSTOM
BGP_ROUTING_MODE: REGIONAL
IPV4_RANGE:
GATEWAY_IPV4:

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network fgt-hasync-vpc --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network fgt-hasync-vpc --allow tcp:22,tcp:3389,icmp

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/fgt-mgmt-vpc].
NAME: fgt-mgmt-vpc
SUBNET_MODE: CUSTOM
BGP_ROUTING_MODE: REGIONAL
IPV4_RANGE:
GATEWAY_IPV4:

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network fgt-mgmt-vpc --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network fgt-mgmt-vpc --allow tcp:22,tcp:3389,icmp

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/subnetworks/ext-sb-euwest1].
NAME: ext-sb-euwest1
REGION: europe-west1
NETWORK: ext-vpc-global
RANGE: 172.20.0.0/24
STACK_TYPE: IPV4_ONLY
IPV6_ACCESS_TYPE:
INTERNAL_IPV6_PREFIX:
EXTERNAL_IPV6_PREFIX:
1748

Creating firewall...working..Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/firewalls/fgt-hasync-fw-allowall].
Creating firewall...done.
NAME: fgt-hasync-fw-allowall
NETWORK: fgt-hasync-vpc
DIRECTION: INGRESS
PRIORITY: 1000
ALLOW: all
DENY:
DISABLED: False
Creating firewall...working..Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/firewalls/fgt-mgmt-fw-allow-admin].
Creating firewall...done.
NAME: fgt-mgmt-fw-allow-admin
NETWORK: fgt-mgmt-vpc
DIRECTION: INGRESS
PRIORITY: 1000
ALLOW: tcp:22,tcp:443
DENY:
DISABLED: False
Creating router [ext-nat-cr-euwest1]...done.   
NAME: ext-nat-cr-euwest1
REGION: europe-west1
NETWORK: ext-vpc-global
Creating NAT [ext-nat-euwest1] in router [ext-nat-cr-euwest1]...done.     
################################################################################
#
# II. Reserve static IP addresses
# -------------------------------
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgt-mgmt-eip-euwest1-b].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgt-mgmt-eip-euwest1-c].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgt-ip-int-euwest1-b].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgt-ip-int-euwest1-c].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgtilb-ip-int-euwest1].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgt-ip-ext-euwest1-b].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgt-ip-ext-euwest1-c].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgt-ip-hasync-euwest1-b].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgt-ip-hasync-euwest1-c].
################################################################################

1753
expected as using payg

New disks are unformatted. You must format and mount a disk before it
can be used. You can find instructions on how to do this at:

https://cloud.google.com/compute/docs/disks/add-persistent-disk#formatting

ERROR: (gcloud.compute.instances.create) Unable to read file [lic1.lic]: [Errno 2] No such file or directory: 'lic1.lic'
ERROR: (gcloud.compute.instances.create) Unable to read file [lic2.lic]: [Errno 2] No such file or directory: 'lic2.lic'
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-b/instanceGroups/fgt-umig-euwest1-b].
NAME: fgt-umig-euwest1-b

ERROR: (gcloud.compute.instance-groups.unmanaged.add-instances) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-b/instances/fgt-vm-euwest1-b' was not found

ERROR: (gcloud.compute.instance-groups.unmanaged.add-instances) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-c/instances/fgt-vm-euwest1-c' was not found

Waiting 2 minutes for the VM instance to bootstrap...

ERROR: (gcloud.compute.instances.describe) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-b/instances/fgt-vm-euwest1-b' was not found

ssh: connect to host 34.79.46.47 port 22: Connection timed out
ls: cannot access '/home/michael/.ssh/id_rsa.pub': No such file or directory
Generating new SSH key
Generating public/private rsa key pair.
Enter file in which to save the key (/home/michael/.ssh/id_rsa):
Created directory '/home/michael/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/michael/.ssh/id_rsa
Your public key has been saved in /home/michael/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:g8q2shAfMo1g5SLjSx2uSHamT1TJ51xenNoxoa12dQE michael@cs-563571566046-default
The key's randomart image is:
+---[RSA 3072]----+
|   .       . E.. |
|  o . .   + o   .|
|+o o + . o B . . |
|=++ o +.o = + .  |
|==o*  .oS= o     |
|==Bo .  ...      |
|o+..+            |
| .+. .           |
|  .+.            |
+----[SHA256]-----+
Uploading new SSH key to FortiGate. Please log in using your new admin password:

will retry with
-  --metadata-from-file="user-data=metadata_active.txt,license=lic1.lic" \
+  --metadata-from-file="user-data=metadata_active.txt" \

michael@cloudshell:~/github/fortigate-tutorial-gcp/gcloud (fortigate-tutorial-gcp-lgz3)$ ./tutorial-delete.sh
ERROR: (gcloud.compute.instances.delete) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-b/instances/wrkld-tier2-web' was not found

ERROR: (gcloud.compute.instances.delete) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-c/instances/fgt-vm-euwest1-c' was not found

Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-b/instanceGroups/fgt-umig-euwest1-b].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-c/instanceGroups/fgt-umig-euwest1-c].

Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/fgt-hasync-vpc].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/fgt-mgmt-vpc].
michael@cloudshell:~/github/fortigate-tutorial-gcp/gcloud (fortigate-tutorial-gcp-lgz3)$

1803

rerunning

obriensystems commented 1 year ago

Running into the 5 quota limit on compute engine api networks - vpc's per project

ERROR: (gcloud.compute.networks.create) Could not fetch resource:
 - Quota 'NETWORKS' exceeded.  Limit: 5.0 globally.

your active configuration is: [cloudshell-25732]
ERROR: (gcloud.iam.roles.create) Resource in projects [fortigate-tutorial-gcp-lgz3] is the subject of a conflict: A role named FortigateSdnReader in projects/fortigate-tutorial-gcp-lgz3 already exists.
ERROR: (gcloud.iam.service-accounts.create) Resource in projects [fortigate-tutorial-gcp-lgz3] is the subject of a conflict: Service account fortigatesdn-ro already exists within project projects/fortigate-tutorial-gcp-lgz3.
- '@type': type.googleapis.com/google.rpc.ResourceInfo

version: 1
################################################################################
#
# IV. Create Fortigate instances
# ------------------------------
ERROR: (gcloud.compute.networks.subnets.describe) Could not fetch resource:
 - The service is currently unavailable.

ERROR: (gcloud.compute.disks.create) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-b/disks/fgt-logdisk-euwest1-b' already exists

ERROR: (gcloud.compute.disks.create) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-c/disks/fgt-logdisk-euwest1-c' already exists

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-b/instances/fgt-vm-euwest1-b].
NAME: fgt-vm-euwest1-b
ZONE: europe-west1-b
MACHINE_TYPE: e2-standard-4
PREEMPTIBLE:
INTERNAL_IP: 172.20.0.2,172.20.1.2,172.20.2.2,172.20.3.2
EXTERNAL_IP: 34.79.46.47
STATUS: RUNNING
./tutorial-create.sh: line 397: --metadata-from-file=user-data=metadata_active.txt: command not found
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-c/instances/fgt-vm-euwest1-c].
NAME: fgt-vm-euwest1-c
ZONE: europe-west1-c
MACHINE_TYPE: e2-standard-4
PREEMPTIBLE:
INTERNAL_IP: 172.20.0.3,172.20.1.3,172.20.2.3,172.20.3.3
EXTERNAL_IP: 34.140.232.93
STATUS: RUNNING
./tutorial-create.sh: line 414: --metadata-from-file=user-data=metadata_passive.txt: command not found

Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-b/instanceGroups/fgt-umig-euwest1-b].
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-c/instanceGroups/fgt-umig-euwest1-c].
Waiting 2 minutes for the VM instance to bootstrap...
####################################################################################
# This script will now attempt to connect to CLI of your newly-deployed FortiGate. #
# Please log in as 'admin' using the instance id printed below as initial password
# and change the password to your own as prompted. When done, please logout using
# 'exit' command to resume the deployment.
#
#
3237622114135385943

ssh: connect to host 34.79.46.47 port 22: Connection timed out
/home/michael/.ssh/id_rsa.pub
Uploading new SSH key to FortiGate. Please log in using your new admin password:
3237622114135385943

ssh: connect to host 34.79.46.47 port 22: Connection timed out

################################################################################
#
# V. Health checks
# ----------------

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/healthChecks/fgt-hcheck-tcp8008].
NAME: fgt-hcheck-tcp8008
PROTOCOL: HTTP
ssh: connect to host 34.79.46.47 port 22: Connection timed out

################################################################################
#
# VI. Internal Load Balancer
# ---------------------------

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/backendServices/fgtilb-int-bes-euwest1].
NAME: fgtilb-int-bes-euwest1
BACKENDS:
PROTOCOL: TCP

Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/backendServices/fgtilb-int-bes-euwest1].
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/backendServices/fgtilb-int-bes-euwest1].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/forwardingRules/fgtilb-int-fwd-euwest1-tcp].

ssh: connect to host 34.79.46.47 port 22: Connection timed out
ssh: connect to host 34.79.46.47 port 22: Connection timed out
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/routes/rt-int-euwest1-default-via-fgt].
NAME: rt-int-euwest1-default-via-fgt
NETWORK: int-vpc-euwest1
DEST_RANGE: 0.0.0.0/0
NEXT_HOP: 172.20.1.4
PRIORITY: 1000

################################################################################
#
# VII. Workload spoke VPC networks
# --------------------------------

################################################################################
#
# VII. Workload spoke VPC networks
# --------------------------------

ERROR: (gcloud.compute.networks.create) Could not fetch resource:
 - Quota 'NETWORKS' exceeded.  Limit: 5.0 globally.

ERROR: (gcloud.compute.networks.create) Could not fetch resource:
 - Quota 'NETWORKS' exceeded.  Limit: 5.0 globally.

ERROR: (gcloud.compute.routes.delete) argument NAME [NAME ...]: Must be specified.
Usage: gcloud compute routes delete NAME [NAME ...] [optional flags]
  optional flags may be  --help

For detailed information on this command and its flags, run:
  gcloud compute routes delete --help
ERROR: (gcloud.compute.routes.delete) argument NAME [NAME ...]: Must be specified.
Usage: gcloud compute routes delete NAME [NAME ...] [optional flags]
  optional flags may be  --help

For detailed information on this command and its flags, run:
  gcloud compute routes delete --help
ERROR: (gcloud.compute.networks.subnets.create) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/global/networks/wrkld-tier1' was not found

ERROR: (gcloud.compute.networks.subnets.create) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/global/networks/wrkld-tier2' was not found

Creating firewall...failed.
ERROR: (gcloud.compute.firewall-rules.create) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/global/networks/wrkld-tier1' was not found

Creating firewall...failed.
ERROR: (gcloud.compute.firewall-rules.create) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/global/networks/wrkld-tier2' was not found

################################################################################
#
# VIII. Peering workloads to trusted VPC network
# ---------------------------------------------

Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/int-vpc-euwest1].
---
autoCreateSubnetworks: false
creationTimestamp: '2022-10-19T17:40:27.018-07:00'
id: '8799527994209936916'
kind: compute#network
name: int-vpc-euwest1
networkFirewallPolicyEnforcementOrder: AFTER_CLASSIC_FIREWALL
peerings:
- autoCreateRoutes: true
  exchangeSubnetRoutes: true
  exportCustomRoutes: true
  exportSubnetRoutesWithPublicIp: true
  importCustomRoutes: false
  importSubnetRoutesWithPublicIp: false
  name: wrkld-peer-hub-to-tier1
  network: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/wrkld-tier1
  state: INACTIVE
  stateDetails: '[2022-10-19T17:59:34.260-07:00]: Waiting for peer network to connect.'
routingConfig:
  routingMode: REGIONAL
selfLink: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/int-vpc-euwest1
selfLinkWithId: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/8799527994209936916
subnetworks:
- https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/subnetworks/int-sb-euwest1
ERROR: (gcloud.compute.networks.peerings.create) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/global/networks/wrkld-tier1' was not found

Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/int-vpc-euwest1].
---
autoCreateSubnetworks: false
creationTimestamp: '2022-10-19T17:40:27.018-07:00'
id: '8799527994209936916'
kind: compute#network
name: int-vpc-euwest1
networkFirewallPolicyEnforcementOrder: AFTER_CLASSIC_FIREWALL
peerings:
- autoCreateRoutes: true
  exchangeSubnetRoutes: true
  exportCustomRoutes: true
  exportSubnetRoutesWithPublicIp: true
  importCustomRoutes: false
  importSubnetRoutesWithPublicIp: false
  name: wrkld-peer-hub-to-tier1
  network: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/wrkld-tier1
  state: INACTIVE
  stateDetails: '[2022-10-19T17:59:34.260-07:00]: Waiting for peer network to connect.'
- autoCreateRoutes: true
  exchangeSubnetRoutes: true
  exportCustomRoutes: true
  exportSubnetRoutesWithPublicIp: true
  importCustomRoutes: false
  importSubnetRoutesWithPublicIp: false
  name: wrkld-peer-hub-to-tier2
  network: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/wrkld-tier2
  state: INACTIVE
  stateDetails: '[2022-10-19T17:59:37.725-07:00]: Waiting for peer network to connect.'
routingConfig:
  routingMode: REGIONAL
selfLink: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/int-vpc-euwest1
selfLinkWithId: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/8799527994209936916
subnetworks:
- https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/subnetworks/int-sb-euwest1
ERROR: (gcloud.compute.networks.peerings.create) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/global/networks/wrkld-tier2' was not found

ssh: connect to host 34.79.46.47 port 22: Connection timed out

################################################################################
#
# IX. External Load Balancer
# ----------------------------

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgtelb-serv1-eip-euwest1].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/backendServices/fgtelb-bes-euwest1].
NAME: fgtelb-bes-euwest1
BACKENDS:
PROTOCOL: UNSPECIFIED
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/backendServices/fgtelb-bes-euwest1].

Created [https://www.googleapis.com/compute/beta/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/forwardingRules/fgtelb-serv1-fwd-euwest1-l3].
ssh: connect to host 34.79.46.47 port 22: Connection timed out

##############################################
Configuring outbound connections
----------------------------------------------

ssh: connect to host 34.79.46.47 port 22: Connection timed out

###############################################
# Sample workload VMs
#----------------------------------------------

ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/subnetworks/wrkld-sb-tier1-euwest1' was not found

ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/subnetworks/wrkld-sb-tier2-euwest1' was not found

#############################################
# Forward Inbound Connections
#--------------------------------------------

ssh: connect to host 34.79.46.47 port 22: Connection timed out

=======================================
# Next step:
# - run tutorial-test.sh to verify everything works
michael@cloudshell:~/github/fortigate-tutorial-gcp/gcloud (fortigate-tutorial-gcp-lgz3)$

2110
delete

michael@cloudshell:~/github/fortigate-tutorial-gcp/gcloud (fortigate-tutorial-gcp-lgz3)$ ./tutorial-delete.sh
ERROR: (gcloud.compute.instances.delete) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-b/instances/wrkld-tier2-web' was not found

ERROR: (gcloud.compute.instances.delete) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-b/instances/wrkld-tier1-proxy' was not found

ERROR: (gcloud.compute.firewall-rules.delete) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/global/firewalls/wrkld-fw-tier1-allowall' was not found

ERROR: (gcloud.compute.firewall-rules.delete) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/global/firewalls/wrkld-fw-tier2-allowall' was not found

Deleted [https://www.googleapis.com/compute/beta/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/forwardingRules/fgtelb-serv1-fwd-euwest1-l3].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgtelb-serv1-eip-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/backendServices/fgtelb-bes-euwest1].
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/int-vpc-euwest1].

Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/int-vpc-euwest1].
ERROR: (gcloud.compute.networks.peerings.delete) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/global/networks/wrkld-tier1' was not found

Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/int-vpc-euwest1].
ERROR: (gcloud.compute.networks.peerings.delete) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/global/networks/wrkld-tier2' was not found

ERROR: (gcloud.compute.networks.subnets.delete) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/subnetworks/wrkld-sb-tier1-euwest1' was not found

ERROR: (gcloud.compute.networks.subnets.delete) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/subnetworks/wrkld-sb-tier2-euwest1' was not found

ERROR: (gcloud.compute.networks.delete) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/global/networks/wrkld-tier1' was not found

ERROR: (gcloud.compute.networks.delete) Could not fetch resource:
 - The resource 'projects/fortigate-tutorial-gcp-lgz3/global/networks/wrkld-tier2' was not found

Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/routes/rt-int-euwest1-default-via-fgt].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/forwardingRules/fgtilb-int-fwd-euwest1-tcp].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/backendServices/fgtilb-int-bes-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/healthChecks/fgt-hcheck-tcp8008].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-b/instances/fgt-vm-euwest1-b].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-c/instances/fgt-vm-euwest1-c].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-b/instanceGroups/fgt-umig-euwest1-b].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/zones/europe-west1-c/instanceGroups/fgt-umig-euwest1-c].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgt-mgmt-eip-euwest1-b].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgt-mgmt-eip-euwest1-c].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgt-ip-int-euwest1-b].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgt-ip-int-euwest1-c].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgtilb-ip-int-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgt-ip-ext-euwest1-b].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgt-ip-ext-euwest1-c].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgt-ip-hasync-euwest1-b].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/addresses/fgt-ip-hasync-euwest1-c].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/routers/ext-nat-cr-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/firewalls/ext-to-fgt-fw-allowall].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/firewalls/int-to-fgt-fw-allowall].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/firewalls/fgt-hasync-fw-allowall].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/firewalls/fgt-mgmt-fw-allow-admin].

Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/subnetworks/ext-sb-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/subnetworks/int-sb-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/subnetworks/fgt-hasync-sb-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/regions/europe-west1/subnetworks/fgt-mgmt-sb-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/ext-vpc-global].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/int-vpc-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/fgt-hasync-vpc].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz3/global/networks/fgt-mgmt-vpc].
michael@cloudshell:~/github/fortigate-tutorial-gcp/gcloud (fortigate-tutorial-gcp-lgz3)$

2116

2121
michael@cloudshell:~/github/fortigate-tutorial-gcp/gcloud (fortigate-tutorial-gcp-lgz4)$ ./tutorial-create.sh
################################################################################
#
# I. VPCs and subnets
# --------------------
API [compute.googleapis.com] not enabled on project [455265533596]. Would you like to enable and retry (this will take a few minutes)? (y/N)?  y

Enabling service [compute.googleapis.com] on project [455265533596]...

Creating firewall...working..Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz4/global/firewalls/fgt-mgmt-fw-allow-admin].
Creating firewall...done.
NAME: fgt-mgmt-fw-allow-admin
NETWORK: fgt-mgmt-vpc
DIRECTION: INGRESS
PRIORITY: 1000
ALLOW: tcp:22,tcp:443
DENY:
DISABLED: False
Creating router [ext-nat-cr-euwest1]...done.   
NAME: ext-nat-cr-euwest1
REGION: europe-west1
NETWORK: ext-vpc-global
Creating NAT [ext-nat-euwest1] in router [ext-nat-cr-euwest1]...done.     

# III. Create FortiGate service account
# -------------------------------------
Your active configuration is: [cloudshell-25732]
WARNING: API is not enabled for permissions: [container.clusters.list, container.nodes.list, container.pods.list, container.services.list]. Please enable the corresponding APIs to use those permissions.

Created role [FortigateSdnReader].

INTERNAL_IP: 172.20.0.2,172.20.1.2,172.20.2.2,172.20.3.2
EXTERNAL_IP: 34.140.242.199
STATUS: RUNNING
./tutorial-create.sh: line 397: --metadata-from-file=user-data=metadata_active.txt: command not found
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz4/zones/europe-west1-c/instances/fgt-vm-euwest1-c].
NAME: fgt-vm-euwest1-c
ZONE: europe-west1-c
MACHINE_TYPE: e2-standard-4
PREEMPTIBLE:
INTERNAL_IP: 172.20.0.3,172.20.1.3,172.20.2.3,172.20.3.3
EXTERNAL_IP: 35.189.219.100
STATUS: RUNNING
./tutorial-create.sh: line 414: --metadata-from-file=user-data=metadata_passive.txt: command not found
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz4/zones/europe-west1-b/instanceGroups/fgt-umig-euwest1-b].
NAME: fgt-umig-euwest1-b
LOCATION: europe-west1-b
SCOPE: zone

################################################################################
#
# VII. Workload spoke VPC networks
# --------------------------------

ERROR: (gcloud.compute.networks.create) Could not fetch resource:
 - Quota 'NETWORKS' exceeded.  Limit: 5.0 globally.

ERROR: (gcloud.compute.networks.create) Could not fetch resource:
 - Quota 'NETWORKS' exceeded.  Limit: 5.0 globally.

2140
2153

https://console.cloud.google.com/iam-admin/quotas?service=compute.googleapis.com&project=fortigate-tutorial-gcp-lgz4

Compute Engine API
Thank you for submitting Case # (ID:1fb983e3860c45839043a35756513239) to Google Cloud Platform support for the following quota:
        Change Networks from 5 to 10
Your request is being processed and you should receive an email confirmation for your request. Should you need further assistance, you can respond to that email. You can also track the status of this request [here](https://console.cloud.google.com/iam-admin/quotas/qirs?project=fortigate-tutorial-gcp-lgz4).

delete default network

# delete service account
GCP_PROJECT_ID=$(gcloud config get-value project)
gcloud iam service-accounts delete fortigatesdn-ro@$GCP_PROJECT_ID.iam.gserviceaccount.com

delete custom role
RROR: (gcloud.iam.roles.create) Resource in projects [fortigate-tutorial-gcp-lgz4] is the subject of a conflict: A role named FortigateSdnReader in projects/fortigate-tutorial-gcp-lgz4 already exi

projects/fortigate-tutorial-gcp-lgz4/roles/FortigateSdnReader

2214 start
- delete the service account first - or add to the delete script along with the internal vpc

2227

vms ok
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz4/zones/europe-west1-b/instanceGroups/fgt-umig-euwest1-b].
NAME: fgt-umig-euwest1-b
LOCATION: europe-west1-b
SCOPE: zone
NETWORK:
MANAGED:
INSTANCES: 0
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz4/zones/europe-west1-c/instanceGroups/fgt-umig-euwest1-c].
NAME: fgt-umig-euwest1-c
LOCATION: europe-west1-c
SCOPE: zone
NETWORK:
MANAGED:
INSTANCES: 0
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz4/zones/europe-west1-b/instanceGroups/fgt-umig-euwest1-b].
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz4/zones/europe-west1-c/instanceGroups/fgt-umig-euwest1-c].
Waiting 2 minutes for the VM instance to bootstrap...

also fix
You don't have required permissions:
        compute.organizations.listAssociations
to view the firewall policies inherited by this project.

with removed
# above sa
  #--metadata-from-file="user-data=metadata_active.txt,license=lic1.lic" \
  #--metadata-from-file="user-data=metadata_active.txt" \

fortigate vm still busted
sh: connect to host 35.189.219.100 port 22: Connection timed out

Testing delete jira
GCP_PROJECT_ID=$(gcloud config get-value project)
gcloud iam service-accounts delete fortigatesdn-ro@$GCP_PROJECT_ID.iam.gserviceaccount.com

gcloud iam roles delete FortigateSdnReader --project=$GCP_PROJECT_ID

Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz4/global/networks/fgt-mgmt-vpc].
Your active configuration is: [cloudshell-25732]
You are about to delete service account [fortigatesdn-ro@fortigate-tutorial-gcp-lgz4.iam.gserviceaccount.com].

Do you want to continue (Y/n)?  y

deleted service account [fortigatesdn-ro@fortigate-tutorial-gcp-lgz4.iam.gserviceaccount.com]
deleted: true
etag: BwXrbl8mG2Q=
includedPermissions:
- compute.instances.list
- compute.zones.list
- container.clusters.list
- container.nodes.list
- container.pods.list
- container.services.list
name: projects/fortigate-tutorial-gcp-lgz4/roles/FortigateSdnReader
title: FortiGate SDN Connector Role (read-only)
obriensystems commented 1 year ago
2253

ERROR: (gcloud.iam.roles.create) FAILED_PRECONDITION: You can't create a role with role_id (FortigateSdnReader) where there is an existing role with that role_id in a deleted state.
Created service account [fortigatesdn-ro].

use
GCP_CUSTOM_ROLE=FortigateSdnReader

all 6 VPC's up, but 2 workload VMs failing

Welcome to Cloud Shell! Type "help" to get started.
To set your Cloud Platform project in this session use “gcloud config set project [PROJECT_ID]”
michael@cloudshell:~$ gcloud config set project fortigate-tutorial-gcp-lgz4
Updated property [core/project].
michael@cloudshell:~ (fortigate-tutorial-gcp-lgz4)$ cd github/fortigate-tutorial-gcp/
michael@cloudshell:~/github/fortigate-tutorial-gcp (fortigate-tutorial-gcp-lgz4)$ git diff
diff --git a/gcloud/tutorial-create.sh b/gcloud/tutorial-create.sh
index bcdc731..dfff628 100755
--- a/gcloud/tutorial-create.sh
+++ b/gcloud/tutorial-create.sh
@@ -176,7 +176,7 @@ EOT

 GCP_PROJECT_ID=$(gcloud config get-value project)

-gcloud iam roles create FortigateSdnReader --project=$GCP_PROJECT_ID \
+gcloud iam roles create $GCP_CUSTOM_ROLE --project=$GCP_PROJECT_ID \
   --title="FortiGate SDN Connector Role (read-only)" \
   --permissions="compute.zones.list,compute.instances.list,container.clusters.list,container.nodes.list,container.pods.list,container.services.list"

@@ -185,7 +185,7 @@ gcloud iam service-accounts create fortigatesdn-ro \

 gcloud projects add-iam-policy-binding $GCP_PROJECT_ID \
   --member="serviceAccount:fortigatesdn-ro@$GCP_PROJECT_ID.iam.gserviceaccount.com" \
-  --role="projects/$GCP_PROJECT_ID/roles/FortigateSdnReader"
+  --role="projects/$GCP_PROJECT_ID/roles/$GCP_CUSTOM_ROLE"

 cat <<EOT
 ################################################################################
@@ -384,7 +384,7 @@ gcloud compute disks create fgt-logdisk-$ZONE2_LABEL --zone=$ZONE2 \
 gcloud compute instances create fgt-vm-$ZONE1_LABEL --zone=$ZONE1 \
   --machine-type=e2-standard-4 \
   --image-project=fortigcp-project-001 \
-  --image-family=fortigate-70-byol \
+  --image-family=fortigate-70-payg \
   --can-ip-forward \
   --network-interface="network=ext-vpc-global,subnet=ext-sb-$REGION_LABEL,no-address,private-network-ip=fgt-ip-ext-$ZONE1_LABEL" \
   --network-interface="network=int-vpc-$REGION_LABEL,subnet=int-sb-$REGION_LABEL,no-address,private-network-ip=fgt-ip-int-$ZONE1_LABEL" \
@@ -392,15 +392,17 @@ gcloud compute instances create fgt-vm-$ZONE1_LABEL --zone=$ZONE1 \
   --network-interface="network=fgt-mgmt-vpc,subnet=fgt-mgmt-sb-$REGION_LABEL,address=fgt-mgmt-eip-$ZONE1_LABEL" \
   --disk="auto-delete=yes,boot=no,device-name=logdisk,mode=rw,name=fgt-logdisk-$ZONE1_LABEL" \
   --tags=fgt \
-  --metadata-from-file="user-data=metadata_active.txt,license=lic1.lic" \
   --service-account=fortigatesdn-ro@$GCP_PROJECT_ID.iam.gserviceaccount.com \
   --scopes=cloud-platform

+# above sa
+  #--metadata-from-file="user-data=metadata_active.txt,license=lic1.lic" \
+  #--metadata-from-file="user-data=metadata_active.txt" \

 gcloud compute instances create fgt-vm-$ZONE2_LABEL --zone=$ZONE2 \
   --machine-type=e2-standard-4 \
   --image-project=fortigcp-project-001 \
-  --image-family=fortigate-70-byol \
+  --image-family=fortigate-70-payg \
   --can-ip-forward \
   --network-interface="network=ext-vpc-global,subnet=ext-sb-$REGION_LABEL,no-address,private-network-ip=fgt-ip-ext-$ZONE2_LABEL" \
   --network-interface="network=int-vpc-$REGION_LABEL,subnet=int-sb-$REGION_LABEL,no-address,private-network-ip=fgt-ip-int-$ZONE2_LABEL" \
@@ -408,10 +410,12 @@ gcloud compute instances create fgt-vm-$ZONE2_LABEL --zone=$ZONE2 \
   --network-interface="network=fgt-mgmt-vpc,subnet=fgt-mgmt-sb-$REGION_LABEL,address=fgt-mgmt-eip-$ZONE2_LABEL" \
   --disk="auto-delete=yes,boot=no,device-name=logdisk,mode=rw,name=fgt-logdisk-$ZONE2_LABEL" \
   --tags=fgt \
-  --metadata-from-file="user-data=metadata_passive.txt,license=lic2.lic" \
   --service-account=fortigatesdn-ro@$GCP_PROJECT_ID.iam.gserviceaccount.com \
   --scopes=cloud-platform

+# above sa
+  #--metadata-from-file="user-data=metadata_passive.txt,license=lic2.lic" \
+  #--metadata-from-file="user-data=metadata_passive.txt" \

 ## Create Unmanaged Instance Groups, which will be used by the load balancers
 gcloud compute instance-groups unmanaged create fgt-umig-$ZONE1_LABEL --zone=$ZONE1
@@ -600,7 +604,7 @@ EOT
 ## Google Cloud supports up to 25 spoke VPCs per firewall NIC. The maximum number of spokes
 ## is 150 (using 6 FortiGate network interfaces for trusted VPCs).

-## Create workload VPC networks
+## Create workload VPC networks - Note 5 vpc(network) quota globally per project - with the two workloads we have 6
 gcloud compute networks create wrkld-tier1 \
   --subnet-mode=custom
 gcloud compute networks create wrkld-tier2 \
diff --git a/gcloud/tutorial-delete.sh b/gcloud/tutorial-delete.sh
index 1d9d6d7..0eeca82 100755
--- a/gcloud/tutorial-delete.sh
+++ b/gcloud/tutorial-delete.sh
@@ -61,3 +61,10 @@ gcloud compute networks delete ext-vpc-global -q
 gcloud compute networks delete int-vpc-$REGION_LABEL -q
 gcloud compute networks delete fgt-hasync-vpc -q
 gcloud compute networks delete fgt-mgmt-vpc -q
+
+# delete service account
+GCP_PROJECT_ID=$(gcloud config get-value project)
+gcloud iam service-accounts delete fortigatesdn-ro@$GCP_PROJECT_ID.iam.gserviceaccount.com -q
+
+# delete custom role
+gcloud iam roles delete $GCP_CUSTOM_ROLE --project=$GCP_PROJECT_ID
\ No newline at end of file
diff --git a/gcloud/tutorial-vars.sh b/gcloud/tutorial-vars.sh
index 63a2c6e..46f4f14 100755
--- a/gcloud/tutorial-vars.sh
+++ b/gcloud/tutorial-vars.sh
@@ -18,3 +18,4 @@ ZONE2=europe-west1-c
 REGION_LABEL=$(echo $REGION | tr -d '-' | sed 's/europe/eu/' | sed 's/australia/au/' | sed 's/northamerica/na/' | sed 's/southamerica/sa/' )
 ZONE1_LABEL=$REGION_LABEL-${ZONE1: -1}
 ZONE2_LABEL=$REGION_LABEL-${ZONE2: -1}
+GCP_CUSTOM_ROLE=FortigateSdnReader2
diff --git a/service_account_create.sh b/service_account_create.sh
old mode 100644
new mode 100755
diff --git a/terraform/day0/main.tf b/terraform/day0/main.tf
index 550804f..368a76e 100644
--- a/terraform/day0/main.tf
+++ b/terraform/day0/main.tf
@@ -28,10 +28,10 @@ module "fortigates" {
      "${var.prefix}sb-mgmt"
    ]

-  license_files   = [
-    "lic1.lic",
-    "lic2.lic"
-  ]
+#  license_files   = [
+#    "lic1.lic",
+#    "lic2.lic"
+#  ]

   # If creating sample VPC Networks in the same configuration - wait for them to be created!
   # Remove this explicit dependency if using your own pre-existing networks.
diff --git a/terraform/day0/terraform.tfvars b/terraform/day0/terraform.tfvars
index bd73283..e493e34 100644
--- a/terraform/day0/terraform.tfvars
+++ b/terraform/day0/terraform.tfvars
@@ -1,3 +1,6 @@
-GCP_PROJECT = "forti-emea-se"
+#GCP_PROJECT = "forti-emea-se"
 GCE_REGION = "europe-west3"
-prefix = "fgt-"
+#prefix = "fgt-"
+GCP_PROJECT = "fortigate-tutorial-gcp-lgz2"
+#GCE_REGION = "northamerica-northeast1"
+prefix = "fgt2-"
diff --git a/terraform/modules/fgcp-ha-ap-lb/main.tf b/terraform/modules/fgcp-ha-ap-lb/main.tf
index c3db3e7..194d556 100644
--- a/terraform/modules/fgcp-ha-ap-lb/main.tf
+++ b/terraform/modules/fgcp-ha-ap-lb/main.tf
@@ -2,7 +2,8 @@
 # an image different than the newest 7.0 BYOL series.
 data "google_compute_image" "fgt_image" {
   project         = "fortigcp-project-001"
-  family          = "fortigate-70-byol"
+  #family          = "fortigate-70-byol"
+  family          = "fortigate-70-payg"
 }

 # Pull information about subnets we will connect to FortiGate instances. Subnets must
@@ -121,7 +122,7 @@ resource "google_compute_instance" "fgt-vm" {

   metadata = {
     user-data            = (count.index == 0 ? local.config_active : local.config_passive )
-    license              = fileexists(var.license_files[count.index]) ? file(var.license_files[count.index]) : null
+    #license              = fileexists(var.license_files[count.index]) ? file(var.license_files[count.index]) : null
   }

   network_interface {
diff --git a/terraform/modules/fgcp-ha-ap-lb/variables.tf b/terraform/modules/fgcp-ha-ap-lb/variables.tf
index 081eaa0..a1bc1c3 100644
--- a/terraform/modules/fgcp-ha-ap-lb/variables.tf
+++ b/terraform/modules/fgcp-ha-ap-lb/variables.tf
@@ -57,11 +57,11 @@ variable api_acl {
   description = "List of CIDRs allowed to connect to FortiGate API (must not be 0.0.0.0/0)"
 }

-variable license_files {
-  type        = list(string)
-  default     = ["",""]
-  description = "List of license (.lic) files to be applied for BYOL instances."
-}
+#variable license_files {
+#  type        = list(string)
+#  default     = ["",""]
+#  description = "List of license (.lic) files to be applied for BYOL instances."
+#}

 variable healthcheck_port {
   type        = number
(END)
Screen Shot 2022-10-20 at 00 15 47 Screen Shot 2022-10-20 at 00 16 14 Screen Shot 2022-10-20 at 00 16 33 Screen Shot 2022-10-20 at 00 17 16 Screen Shot 2022-10-20 at 00 17 31 Screen Shot 2022-10-20 at 00 18 56 Screen Shot 2022-10-20 at 00 20 35
fmichaelobrien commented 1 year ago

compute.networks Quota is per project 5-10

Screen Shot 2022-10-20 at 7 54 30 AM

5

Screen Shot 2022-10-20 at 7 53 18 AM
bartekmo commented 1 year ago

Good afternoon team,. I will be reviewing this your implementation. Thank you for the excellent work. Keeping this issue open for initial discussion on config/deploy/use items. /Michael

Hi Michael, thanks for looking into it and for very detailed report. I've got the terraform part fixed (broken reference to subnets). Just waiting for the internal review and merge. gcloud will require a bit more work to troubleshoot but will be there soon

fmichaelobrien commented 1 year ago

LB's come up, but I need to work out the issue with PAYG on the fortigates. Switching region from default europe-west1 to northamerica-northeast1 for data residency checking Will also need to get a hold of 30d licenses as about a week will deplete the $500 credits

delete previous
michael@cloudshell:~$ gcloud config set project fortigate-tutorial-gcp-lgz5                                                                                                                                       
Updated property [core/project].
michael@cloudshell:~ (fortigate-tutorial-gcp-lgz5)$ cd github/fortigate-tutorial-gcp/gcloud/
michael@cloudshell:~/github/fortigate-tutorial-gcp/gcloud (fortigate-tutorial-gcp-lgz5)$ ./tutorial-delete.sh
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/europe-west1-b/instances/wrkld-tier2-web].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/europe-west1-b/instances/wrkld-tier1-proxy].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/firewalls/wrkld-fw-tier1-allowall].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/firewalls/wrkld-fw-tier2-allowall].
Deleted [https://www.googleapis.com/compute/beta/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/forwardingRules/fgtelb-serv1-fwd-euwest1-l3].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/addresses/fgtelb-serv1-eip-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/backendServices/fgtelb-bes-euwest1].
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/int-vpc-euwest1].
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/wrkld-tier1].
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/int-vpc-euwest1].
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/wrkld-tier2].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/subnetworks/wrkld-sb-tier1-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/subnetworks/wrkld-sb-tier2-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/wrkld-tier1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/wrkld-tier2].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/routes/rt-int-euwest1-default-via-fgt].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/forwardingRules/fgtilb-int-fwd-euwest1-tcp].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/backendServices/fgtilb-int-bes-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/healthChecks/fgt-hcheck-tcp8008].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/europe-west1-b/instances/fgt-vm-euwest1-b].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/europe-west1-c/instances/fgt-vm-euwest1-c].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/europe-west1-b/instanceGroups/fgt-umig-euwest1-b].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/europe-west1-c/instanceGroups/fgt-umig-euwest1-c].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/addresses/fgt-mgmt-eip-euwest1-b].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/addresses/fgt-mgmt-eip-euwest1-c].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/addresses/fgt-ip-int-euwest1-b].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/addresses/fgt-ip-int-euwest1-c].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/addresses/fgtilb-ip-int-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/addresses/fgt-ip-ext-euwest1-b].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/addresses/fgt-ip-ext-euwest1-c].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/addresses/fgt-ip-hasync-euwest1-b].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/addresses/fgt-ip-hasync-euwest1-c].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/routers/ext-nat-cr-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/firewalls/ext-to-fgt-fw-allowall].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/firewalls/int-to-fgt-fw-allowall].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/firewalls/fgt-hasync-fw-allowall].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/firewalls/fgt-mgmt-fw-allow-admin].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/subnetworks/ext-sb-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/subnetworks/int-sb-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/subnetworks/fgt-hasync-sb-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/europe-west1/subnetworks/fgt-mgmt-sb-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/ext-vpc-global].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/int-vpc-euwest1].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/fgt-hasync-vpc].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/fgt-mgmt-vpc].
Your active configuration is: [cloudshell-29933]
deleted service account [fortigatesdn-ro@fortigate-tutorial-gcp-lgz5.iam.gserviceaccount.com]
michael@cloudshell:~/github/fortigate-tutorial-gcp/gcloud (fortigate-tutorial-gcp-lgz5)$

#REGION=europe-west1
#ZONE1=europe-west1-b
#ZONE2=europe-west1-c
REGION=northamerica-northeast1
ZONE1=northamerica-northeast1-b
ZONE2=northamerica-northeast1-c

recreate 21:40

michael@cloudshell:~/github/fortigate-tutorial-gcp/gcloud (fortigate-tutorial-gcp-lgz5)$ ./tutorial-create.sh
################################################################################
#
# I. VPCs and subnets
# --------------------
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/ext-vpc-global].
NAME: ext-vpc-global
SUBNET_MODE: CUSTOM
BGP_ROUTING_MODE: REGIONAL
IPV4_RANGE:
GATEWAY_IPV4:

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network ext-vpc-global --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network ext-vpc-global --allow tcp:22,tcp:3389,icmp

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/int-vpc-nanortheast1].
NAME: int-vpc-nanortheast1
SUBNET_MODE: CUSTOM
BGP_ROUTING_MODE: REGIONAL
IPV4_RANGE:
GATEWAY_IPV4:

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network int-vpc-nanortheast1 --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network int-vpc-nanortheast1 --allow tcp:22,tcp:3389,icmp

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/fgt-hasync-vpc].
NAME: fgt-hasync-vpc
SUBNET_MODE: CUSTOM
BGP_ROUTING_MODE: REGIONAL
IPV4_RANGE:
GATEWAY_IPV4:

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network fgt-hasync-vpc --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network fgt-hasync-vpc --allow tcp:22,tcp:3389,icmp

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/fgt-mgmt-vpc].
NAME: fgt-mgmt-vpc
SUBNET_MODE: CUSTOM
BGP_ROUTING_MODE: REGIONAL
IPV4_RANGE:
GATEWAY_IPV4:

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network fgt-mgmt-vpc --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network fgt-mgmt-vpc --allow tcp:22,tcp:3389,icmp

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/subnetworks/ext-sb-nanortheast1].
NAME: ext-sb-nanortheast1
REGION: northamerica-northeast1
NETWORK: ext-vpc-global
RANGE: 172.20.0.0/24
STACK_TYPE: IPV4_ONLY
IPV6_ACCESS_TYPE:
INTERNAL_IPV6_PREFIX:
EXTERNAL_IPV6_PREFIX:
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/subnetworks/int-sb-nanortheast1].
NAME: int-sb-nanortheast1
REGION: northamerica-northeast1
NETWORK: int-vpc-nanortheast1
RANGE: 172.20.1.0/24
STACK_TYPE: IPV4_ONLY
IPV6_ACCESS_TYPE:
INTERNAL_IPV6_PREFIX:
EXTERNAL_IPV6_PREFIX:
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/subnetworks/fgt-hasync-sb-nanortheast1].
NAME: fgt-hasync-sb-nanortheast1
REGION: northamerica-northeast1
NETWORK: fgt-hasync-vpc
RANGE: 172.20.2.0/24
STACK_TYPE: IPV4_ONLY
IPV6_ACCESS_TYPE:
INTERNAL_IPV6_PREFIX:
EXTERNAL_IPV6_PREFIX:
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/subnetworks/fgt-mgmt-sb-nanortheast1].
NAME: fgt-mgmt-sb-nanortheast1
REGION: northamerica-northeast1
NETWORK: fgt-mgmt-vpc
RANGE: 172.20.3.0/24
STACK_TYPE: IPV4_ONLY
IPV6_ACCESS_TYPE:
INTERNAL_IPV6_PREFIX:
EXTERNAL_IPV6_PREFIX:
Creating firewall...working..Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/firewalls/ext-to-fgt-fw-allowall].
Creating firewall...done.
NAME: ext-to-fgt-fw-allowall
NETWORK: ext-vpc-global
DIRECTION: INGRESS
PRIORITY: 1000
ALLOW: all
DENY:
DISABLED: False
Creating firewall...working..Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/firewalls/int-to-fgt-fw-allowall].
Creating firewall...done.
NAME: int-to-fgt-fw-allowall
NETWORK: int-vpc-nanortheast1
DIRECTION: INGRESS
PRIORITY: 1000
ALLOW: all
DENY:
DISABLED: False
Creating firewall...working..Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/firewalls/fgt-hasync-fw-allowall].
Creating firewall...done.
NAME: fgt-hasync-fw-allowall
NETWORK: fgt-hasync-vpc
DIRECTION: INGRESS
PRIORITY: 1000
ALLOW: all
DENY:
DISABLED: False
Creating firewall...working..Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/firewalls/fgt-mgmt-fw-allow-admin].
Creating firewall...done.
NAME: fgt-mgmt-fw-allow-admin
NETWORK: fgt-mgmt-vpc
DIRECTION: INGRESS
PRIORITY: 1000
ALLOW: tcp:22,tcp:443
DENY:
DISABLED: False
Creating router [ext-nat-cr-nanortheast1]...done.   
NAME: ext-nat-cr-nanortheast1
REGION: northamerica-northeast1
NETWORK: ext-vpc-global
Creating NAT [ext-nat-nanortheast1] in router [ext-nat-cr-nanortheast1]...done.     
################################################################################
#
# II. Reserve static IP addresses
# -------------------------------
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/addresses/fgt-mgmt-eip-nanortheast1-b].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/addresses/fgt-mgmt-eip-nanortheast1-c].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/addresses/fgt-ip-int-nanortheast1-b].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/addresses/fgt-ip-int-nanortheast1-c].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/addresses/fgtilb-ip-int-nanortheast1].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/addresses/fgt-ip-ext-nanortheast1-b].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/addresses/fgt-ip-ext-nanortheast1-c].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/addresses/fgt-ip-hasync-nanortheast1-b].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/addresses/fgt-ip-hasync-nanortheast1-c].
################################################################################
#
# III. Create FortiGate service account
# -------------------------------------
Your active configuration is: [cloudshell-29933]
WARNING: API is not enabled for permissions: [container.clusters.list, container.nodes.list, container.pods.list, container.services.list]. Please enable the corresponding APIs to use those permissions.

ERROR: (gcloud.iam.roles.create) Resource in projects [fortigate-tutorial-gcp-lgz5] is the subject of a conflict: A role named FortigateSdnReader5a in projects/fortigate-tutorial-gcp-lgz5 already exists.
Created service account [fortigatesdn-ro].
Updated IAM policy for project [fortigate-tutorial-gcp-lgz5].
bindings:
- members:
  - deleted:serviceAccount:fortigatesdn-ro@fortigate-tutorial-gcp-lgz5.iam.gserviceaccount.com?uid=107403595753799001107
  - deleted:serviceAccount:fortigatesdn-ro@fortigate-tutorial-gcp-lgz5.iam.gserviceaccount.com?uid=111854150690371374269
  - deleted:serviceAccount:fortigatesdn-ro@fortigate-tutorial-gcp-lgz5.iam.gserviceaccount.com?uid=116273492124683320726
  - serviceAccount:fortigatesdn-ro@fortigate-tutorial-gcp-lgz5.iam.gserviceaccount.com
  role: projects/fortigate-tutorial-gcp-lgz5/roles/FortigateSdnReader5a
- members:
  - serviceAccount:service-172714963729@compute-system.iam.gserviceaccount.com
  role: roles/compute.serviceAgent
- members:
  - serviceAccount:172714963729-compute@developer.gserviceaccount.com
  - serviceAccount:172714963729@cloudservices.gserviceaccount.com
  role: roles/editor
- members:
  - user:michael@landing.gcp.zone
  role: roles/owner
etag: BwXr0gdXo_Y=
version: 1
################################################################################
#
# IV. Create Fortigate instances
# ------------------------------
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/northamerica-northeast1-b/disks/fgt-logdisk-nanortheast1-b].
NAME: fgt-logdisk-nanortheast1-b
ZONE: northamerica-northeast1-b
SIZE_GB: 100
TYPE: pd-ssd
STATUS: READY

New disks are unformatted. You must format and mount a disk before it
can be used. You can find instructions on how to do this at:

https://cloud.google.com/compute/docs/disks/add-persistent-disk#formatting

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/northamerica-northeast1-c/disks/fgt-logdisk-nanortheast1-c].
NAME: fgt-logdisk-nanortheast1-c
ZONE: northamerica-northeast1-c
SIZE_GB: 100
TYPE: pd-ssd
STATUS: READY

New disks are unformatted. You must format and mount a disk before it
can be used. You can find instructions on how to do this at:

https://cloud.google.com/compute/docs/disks/add-persistent-disk#formatting

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/northamerica-northeast1-b/instances/fgt-vm-nanortheast1-b].
NAME: fgt-vm-nanortheast1-b
ZONE: northamerica-northeast1-b
MACHINE_TYPE: e2-standard-4
PREEMPTIBLE:
INTERNAL_IP: 172.20.0.2,172.20.1.2,172.20.2.2,172.20.3.2
EXTERNAL_IP: 34.152.28.131
STATUS: RUNNING
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/northamerica-northeast1-c/instances/fgt-vm-nanortheast1-c].
NAME: fgt-vm-nanortheast1-c
ZONE: northamerica-northeast1-c
MACHINE_TYPE: e2-standard-4
PREEMPTIBLE:
INTERNAL_IP: 172.20.0.3,172.20.1.3,172.20.2.3,172.20.3.3
EXTERNAL_IP: 34.95.28.181
STATUS: RUNNING
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/northamerica-northeast1-b/instanceGroups/fgt-umig-nanortheast1-b].
NAME: fgt-umig-nanortheast1-b
LOCATION: northamerica-northeast1-b
SCOPE: zone
NETWORK:
MANAGED:
INSTANCES: 0
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/northamerica-northeast1-c/instanceGroups/fgt-umig-nanortheast1-c].
NAME: fgt-umig-nanortheast1-c
LOCATION: northamerica-northeast1-c
SCOPE: zone
NETWORK:
MANAGED:
INSTANCES: 0
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/northamerica-northeast1-b/instanceGroups/fgt-umig-nanortheast1-b].
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/northamerica-northeast1-c/instanceGroups/fgt-umig-nanortheast1-c].
Waiting 2 minutes for the VM instance to bootstrap...
####################################################################################
# This script will now attempt to connect to CLI of your newly-deployed FortiGate. #
# Please log in as 'admin' using the instance id printed below as initial password
# and change the password to your own as prompted. When done, please logout using
# 'exit' command to resume the deployment.
#
#
5666257143124885974
ssh: connect to host 34.152.28.131 port 22: Connection timed out
/home/michael/.ssh/id_rsa.pub
Uploading new SSH key to FortiGate. Please log in using your new admin password:
ssh: connect to host 34.152.28.131 port 22: Connection timed out

################################################################################
#
# V. Health checks
# ----------------

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/healthChecks/fgt-hcheck-tcp8008].
NAME: fgt-hcheck-tcp8008
PROTOCOL: HTTP
ssh: connect to host 34.152.28.131 port 22: Connection timed out

################################################################################
#
# VI. Internal Load Balancer
# ---------------------------

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/backendServices/fgtilb-int-bes-nanortheast1].
NAME: fgtilb-int-bes-nanortheast1
BACKENDS:
PROTOCOL: TCP
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/backendServices/fgtilb-int-bes-nanortheast1].
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/backendServices/fgtilb-int-bes-nanortheast1].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/forwardingRules/fgtilb-int-fwd-nanortheast1-tcp].
ssh: connect to host 34.152.28.131 port 22: Connection timed out
ssh: connect to host 34.152.28.131 port 22: Connection timed out
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/routes/rt-int-nanortheast1-default-via-fgt].
NAME: rt-int-nanortheast1-default-via-fgt
NETWORK: int-vpc-nanortheast1
DEST_RANGE: 0.0.0.0/0
NEXT_HOP: 172.20.1.4
PRIORITY: 1000

################################################################################
#
# VII. Workload spoke VPC networks
# --------------------------------

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/wrkld-tier1].
NAME: wrkld-tier1
SUBNET_MODE: CUSTOM
BGP_ROUTING_MODE: REGIONAL
IPV4_RANGE:
GATEWAY_IPV4:

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network wrkld-tier1 --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network wrkld-tier1 --allow tcp:22,tcp:3389,icmp

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/wrkld-tier2].
NAME: wrkld-tier2
SUBNET_MODE: CUSTOM
BGP_ROUTING_MODE: REGIONAL
IPV4_RANGE:
GATEWAY_IPV4:

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network wrkld-tier2 --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network wrkld-tier2 --allow tcp:22,tcp:3389,icmp

Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/routes/default-route-0c346d9fe5fdb63b].
Deleted [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/routes/default-route-ef9d013ed61a60a6].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/subnetworks/wrkld-sb-tier1-nanortheast1].
NAME: wrkld-sb-tier1-nanortheast1
REGION: northamerica-northeast1
NETWORK: wrkld-tier1
RANGE: 10.0.0.0/16
STACK_TYPE: IPV4_ONLY
IPV6_ACCESS_TYPE:
INTERNAL_IPV6_PREFIX:
EXTERNAL_IPV6_PREFIX:
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/subnetworks/wrkld-sb-tier2-nanortheast1].
NAME: wrkld-sb-tier2-nanortheast1
REGION: northamerica-northeast1
NETWORK: wrkld-tier2
RANGE: 10.1.0.0/16
STACK_TYPE: IPV4_ONLY
IPV6_ACCESS_TYPE:
INTERNAL_IPV6_PREFIX:
EXTERNAL_IPV6_PREFIX:
Creating firewall...working..Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/firewalls/wrkld-fw-tier1-allowall].
Creating firewall...done.
NAME: wrkld-fw-tier1-allowall
NETWORK: wrkld-tier1
DIRECTION: INGRESS
PRIORITY: 1000
ALLOW: all
DENY:
DISABLED: False
Creating firewall...working..Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/firewalls/wrkld-fw-tier2-allowall].
Creating firewall...done.
NAME: wrkld-fw-tier2-allowall
NETWORK: wrkld-tier2
DIRECTION: INGRESS
PRIORITY: 1000
ALLOW: all
DENY:
DISABLED: False

################################################################################
#
# VIII. Peering workloads to trusted VPC network
# ---------------------------------------------

Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/int-vpc-nanortheast1].
---
autoCreateSubnetworks: false
creationTimestamp: '2022-10-24T18:39:55.168-07:00'
id: '7169436469125873796'
kind: compute#network
name: int-vpc-nanortheast1
networkFirewallPolicyEnforcementOrder: AFTER_CLASSIC_FIREWALL
peerings:
- autoCreateRoutes: true
  exchangeSubnetRoutes: true
  exportCustomRoutes: true
  exportSubnetRoutesWithPublicIp: true
  importCustomRoutes: false
  importSubnetRoutesWithPublicIp: false
  name: wrkld-peer-hub-to-tier1
  network: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/wrkld-tier1
  state: INACTIVE
  stateDetails: '[2022-10-24T19:00:19.627-07:00]: Waiting for peer network to connect.'
routingConfig:
  routingMode: REGIONAL
selfLink: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/int-vpc-nanortheast1
selfLinkWithId: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/7169436469125873796
subnetworks:
- https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/subnetworks/int-sb-nanortheast1
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/wrkld-tier1].
---
autoCreateSubnetworks: false
creationTimestamp: '2022-10-24T18:58:50.303-07:00'
id: '3541536310692609077'
kind: compute#network
name: wrkld-tier1
networkFirewallPolicyEnforcementOrder: AFTER_CLASSIC_FIREWALL
peerings:
- autoCreateRoutes: true
  exchangeSubnetRoutes: true
  exportCustomRoutes: false
  exportSubnetRoutesWithPublicIp: true
  importCustomRoutes: true
  importSubnetRoutesWithPublicIp: false
  name: wrkld-peer-tier1-to-hub
  network: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/int-vpc-nanortheast1
  state: ACTIVE
  stateDetails: '[2022-10-24T19:00:24.785-07:00]: Connected.'
routingConfig:
  routingMode: REGIONAL
selfLink: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/wrkld-tier1
selfLinkWithId: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/3541536310692609077
subnetworks:
- https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/subnetworks/wrkld-sb-tier1-nanortheast1
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/int-vpc-nanortheast1].
---
autoCreateSubnetworks: false
creationTimestamp: '2022-10-24T18:39:55.168-07:00'
id: '7169436469125873796'
kind: compute#network
name: int-vpc-nanortheast1
networkFirewallPolicyEnforcementOrder: AFTER_CLASSIC_FIREWALL
peerings:
- autoCreateRoutes: true
  exchangeSubnetRoutes: true
  exportCustomRoutes: true
  exportSubnetRoutesWithPublicIp: true
  importCustomRoutes: false
  importSubnetRoutesWithPublicIp: false
  name: wrkld-peer-hub-to-tier1
  network: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/wrkld-tier1
  state: ACTIVE
  stateDetails: '[2022-10-24T19:00:24.785-07:00]: Connected.'
- autoCreateRoutes: true
  exchangeSubnetRoutes: true
  exportCustomRoutes: true
  exportSubnetRoutesWithPublicIp: true
  importCustomRoutes: false
  importSubnetRoutesWithPublicIp: false
  name: wrkld-peer-hub-to-tier2
  network: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/wrkld-tier2
  state: INACTIVE
  stateDetails: '[2022-10-24T19:00:38.920-07:00]: Waiting for peer network to connect.'
routingConfig:
  routingMode: REGIONAL
selfLink: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/int-vpc-nanortheast1
selfLinkWithId: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/7169436469125873796
subnetworks:
- https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/subnetworks/int-sb-nanortheast1
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/wrkld-tier2].
---
autoCreateSubnetworks: false
creationTimestamp: '2022-10-24T18:59:06.227-07:00'
id: '5236226089528523781'
kind: compute#network
name: wrkld-tier2
networkFirewallPolicyEnforcementOrder: AFTER_CLASSIC_FIREWALL
peerings:
- autoCreateRoutes: true
  exchangeSubnetRoutes: true
  exportCustomRoutes: false
  exportSubnetRoutesWithPublicIp: true
  importCustomRoutes: true
  importSubnetRoutesWithPublicIp: false
  name: wrkld-peer-tier2-to-hub
  network: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/int-vpc-nanortheast1
  state: ACTIVE
  stateDetails: '[2022-10-24T19:00:43.132-07:00]: Connected.'
routingConfig:
  routingMode: REGIONAL
selfLink: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/wrkld-tier2
selfLinkWithId: https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/global/networks/5236226089528523781
subnetworks:
- https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/subnetworks/wrkld-sb-tier2-nanortheast1
ssh: connect to host 34.152.28.131 port 22: Connection timed out

################################################################################
#
# IX. External Load Balancer
# ----------------------------

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/addresses/fgtelb-serv1-eip-nanortheast1].
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/backendServices/fgtelb-bes-nanortheast1].
NAME: fgtelb-bes-nanortheast1
BACKENDS:
PROTOCOL: UNSPECIFIED

Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/backendServices/fgtelb-bes-nanortheast1].
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/backendServices/fgtelb-bes-nanortheast1].
Created [https://www.googleapis.com/compute/beta/projects/fortigate-tutorial-gcp-lgz5/regions/northamerica-northeast1/forwardingRules/fgtelb-serv1-fwd-nanortheast1-l3].
ssh: connect to host 34.152.28.131 port 22: Connection timed out

##############################################
Configuring outbound connections
----------------------------------------------

ssh: connect to host 34.152.28.131 port 22: Connection timed out

###############################################
# Sample workload VMs
#----------------------------------------------

Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/northamerica-northeast1-b/instances/wrkld-tier1-proxy].
NAME: wrkld-tier1-proxy
ZONE: northamerica-northeast1-b
MACHINE_TYPE: e2-small
PREEMPTIBLE:
INTERNAL_IP: 10.0.0.5
EXTERNAL_IP:
STATUS: RUNNING
Created [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/northamerica-northeast1-b/instances/wrkld-tier2-web].
NAME: wrkld-tier2-web
ZONE: northamerica-northeast1-b
MACHINE_TYPE: e2-small
PREEMPTIBLE:
INTERNAL_IP: 10.1.0.5
EXTERNAL_IP:
STATUS: RUNNING

#############################################
# Forward Inbound Connections
#--------------------------------------------

ssh: connect to host 34.152.28.131 port 22: Connection timed out

=======================================
# Next step:
# - run tutorial-test.sh to verify everything works

2215

Screen Shot 2022-10-24 at 10 15 19 PM Screen Shot 2022-10-24 at 10 14 59 PM
fmichaelobrien commented 1 year ago
Screen Shot 2022-10-24 at 10 57 27 PM Screen Shot 2022-10-24 at 10 58 13 PM Screen Shot 2022-10-24 at 10 58 34 PM Screen Shot 2022-10-24 at 10 59 00 PM Screen Shot 2022-10-24 at 10 59 24 PM Screen Shot 2022-10-24 at 10 59 40 PM Screen Shot 2022-10-24 at 10 59 52 PM Screen Shot 2022-10-24 at 11 00 17 PM Screen Shot 2022-10-24 at 11 00 29 PM Screen Shot 2022-10-24 at 11 01 19 PM Screen Shot 2022-10-24 at 11 01 42 PM Screen Shot 2022-10-24 at 11 01 56 PM
bartekmo commented 1 year ago

Note that the custom role and the service account are not deleted on purpose - there's a week (?) retention period (like for projects), so they cannot be simply dropped and immediately recreated.

fmichaelobrien commented 1 year ago

Found that out (not really deleted - just marked) and commented the code

# delete service account
GCP_PROJECT_ID=$(gcloud config get-value project)
gcloud iam service-accounts delete fortigatesdn-ro@$GCP_PROJECT_ID.iam.gserviceaccount.com -q

# delete custom role (only marks as deleted) - ignore the "exists" warning on a retry
#gcloud iam roles delete $GCP_CUSTOM_ROLE --project=$GCP_PROJECT_ID

I am still working out the fortigate VM boostrap for PAYG in the gcloud deploy currently running with 395 commented https://github.com/fortinet/fortigate-tutorial-gcp/blob/main/gcloud/tutorial-create.sh#L395

tracking via #3

  --image-project=fortigcp-project-001 \
  --image-family=fortigate-70-payg \

  #--metadata-from-file="user-data=metadata_active.txt,license=lic1.lic" \
  #--metadata-from-file="user-data=metadata_active.txt" \

logs

INSTANCES: 0
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/northamerica-northeast1-b/instanceGroups/fgt-umig-nanortheast1-b].
Updated [https://www.googleapis.com/compute/v1/projects/fortigate-tutorial-gcp-lgz5/zones/northamerica-northeast1-c/instanceGroups/fgt-umig-nanortheast1-c].
Waiting 2 minutes for the VM instance to bootstrap...
####################################################################################
# This script will now attempt to connect to CLI of your newly-deployed FortiGate. #
# Please log in as 'admin' using the instance id printed below as initial password
# and change the password to your own as prompted. When done, please logout using
# 'exit' command to resume the deployment.
#
#
5666257143124885974
ssh: connect to host 34.152.28.131 port 22: Connection timed out
/home/michael/.ssh/id_rsa.pub
Uploading new SSH key to FortiGate. Please log in using your new admin password:
ssh: connect to host 34.152.28.131 port 22: Connection timed out

checking that I have billing account admin first on this test org (landing.gcp.zone) first

fmichaelobrien commented 1 year ago

Update: using BYOL for now - thank you Fortinet as usual I will look at marketplace roles later login change must be done within 10 sec

All good on the gcloud deployment - thank you team

fgt-vm-nanortheast1-b #
=======================================
# Next step:
# - run tutorial-test.sh to verify everything works
michael@cloudshell:~/github/fortigate-tutorial-gcp/gcloud (fortigate-tutorial-gcp-lgz5)$ ./tutorial-test.sh
------------------------------------------------------------------------------------
# This script will run a series of tests to verify if your deployment works correctly.
# With each test you will see information about the expected output - verify if it's
# matching what is returned by the test commands.
------------------------------------------------------------------------------------

-----------------------------------------------------------
##  TEST: FGT HA clustering and licensing
##  Expected output: primary and secondary reported with proper hostnames and non-empty serial numbers
fgt-vm-nanortheast1-b # HA Health Status: OK
Primary     : fgt-vm-nanortheast1-b, FGVM08TM22004699, HA cluster index = 1
Secondary   : fgt-vm-nanortheast1-c, FGVM08TM22004700, HA cluster index = 0
fgt-vm-nanortheast1-b #
-----------------------------------------------------------
##  TEST: ELB health
##  Expected output: one healthy, one unhealthy backend
{
  "ipAddress": "172.20.0.2",
  "healthState": "HEALTHY"
}
{
  "ipAddress": "172.20.0.3",
  "healthState": "UNHEALTHY"
}
-----------------------------------------------------------
##  TEST: ILB trusted health
##  Expected output: one healthy, one unhealthy backend
{
  "ipAddress": "172.20.1.2",
  "healthState": "HEALTHY"
}
{
  "ipAddress": "172.20.1.3",
  "healthState": "UNHEALTHY"
}
-----------------------------------------------------------
##  TEST: peering routes for wrkld-tier1
##  Expected output: STATIC_PEERING_ROUTE to 0.0.0.0 is listed as accepted
DEST_RANGE: 172.20.1.0/24
TYPE: SUBNET_PEERING_ROUTE
NEXT_HOP_REGION: northamerica-northeast1
PRIORITY: 0
STATUS: accepted

DEST_RANGE: 0.0.0.0/0
TYPE: STATIC_PEERING_ROUTE
NEXT_HOP_REGION: northamerica-northeast1
PRIORITY: 1000
STATUS: accepted
-----------------------------------------------------------
##  TEST: peering routes for wrkld-tier2
##  Expected output: STATIC_PEERING_ROUTE to 0.0.0.0 is listed as accepted
DEST_RANGE: 172.20.1.0/24
TYPE: SUBNET_PEERING_ROUTE
NEXT_HOP_REGION: northamerica-northeast1
PRIORITY: 0
STATUS: accepted

DEST_RANGE: 0.0.0.0/0
TYPE: STATIC_PEERING_ROUTE
NEXT_HOP_REGION: northamerica-northeast1
PRIORITY: 1000
STATUS: accepted
-----------------------------------------------------------
##  TEST: website working
##  Expected output: HTTP 200 OK headers from nginx server
HTTP/1.1 200 OK
Server: nginx/1.18.0
Date: Tue, 01 Nov 2022 13:38:31 GMT
Content-Type: text/html
Content-Length: 612
Connection: keep-alive
Last-Modified: Tue, 01 Nov 2022 13:36:31 GMT
ETag: "636120df-264"
Accept-Ranges: bytes

-----------------------------------------------------------
##  TEST: website protected
##  Expected output: information about blocked access to EICAR_TEST_FILE virus
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4810  100  4810    0     0  84385      0 --:--:-- --:--:-- --:--:-- 84385
    <h1>High Security Alert</h1>
    <p>You are not permitted to download the file "eicar.com" because it is infected with the virus "EICAR_TEST_FILE".</p>

========================================
# Next step:
# - open http://35....3 to open protected web page
# - open https://35..3 to explore your FortiGate
# - run tutorial-delete.sh to clean up
michael@cloudshell:~/github/fortigate-tutorial-gcp/gcloud (fortigate-tutorial-gcp-lgz5)$ ^C
fmichaelobrien commented 1 year ago
Screen Shot 2022-11-01 at 10 05 23 AM
fmichaelobrien commented 1 year ago

AV EICAR test and IDS works well

Screen Shot 2022-11-02 at 9 42 48 AM Screen Shot 2022-11-02 at 9 43 13 AM