Data source google_compute_network does not support shared VPC

lwatterworth-cldr commented 4 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.
If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v0.12.28 (also tried 0.13.3)

provider.google v3.43.0

Affected Resource(s)

google_compute_network data source (possibly others)

Terraform Configuration Files

locals {
  google_project  = "gcp-sharedvpc-service-project"
  google_region   = "us-west2"
  shared_vpc_name = "projects/gcp-sharedvpc-host-project/global/networks/shared-vpc"
  #shared_vpc_name = "shared-vpc"
}

provider "google" {
  project = local.google_project
  region  = local.google_region
}

data "google_compute_network" "shared_vpc" {
  name    = local.shared_vpc_name
  project = local.google_project
}

output "data_google_compute_network_shared_vpc" {
  value = data.google_compute_network.shared_vpc
}

Debug Output

(snip)
2020-10-15T18:03:19.694-0400 [DEBUG] plugin.terraform-provider-google_v3.43.0_x5: ---[ REQUEST ]---------------------------------------
2020-10-15T18:03:19.694-0400 [DEBUG] plugin.terraform-provider-google_v3.43.0_x5: GET /compute/v1/projects/gcp-sharedvpc-service-project/global/networks/projects%2Fgcp-sharedvpc-host-project%2Fglobal%2Fnetworks%2Fshared-vpc?alt=json&prettyPrint=false HTTP/1.1
(snip)
2020-10-15T18:03:20.560-0400 [DEBUG] plugin.terraform-provider-google_v3.43.0_x5: ---[ RESPONSE ]--------------------------------------
2020-10-15T18:03:20.560-0400 [DEBUG] plugin.terraform-provider-google_v3.43.0_x5: HTTP/1.1 400 Bad Request
(snip)
'network': 'projects/gcp-sharedvpc-host-project/global/networks/shared-vpc'. Must be a match of regex '[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?|[1-9][0-9]{0,19}'",

Expected Behavior

While working in the service project, expecting to reference the shared VPC by it's full path name, as you can do with the gcloud CLI.

$ gcloud --project gcp-sharedvpc-service-project compute networks describe projects/gcp-sharedvpc-host-project/global/networks/shared-vpc
autoCreateSubnetworks: false
creationTimestamp: '2020-10-15T13:49:35.261-07:00'
id: '5436519676448672560'
kind: compute#network
name: shared-vpc
routingConfig:
  routingMode: GLOBAL
selfLink: https://www.googleapis.com/compute/v1/projects/gcp-sharedvpc-host-project/global/networks/shared-vpc
subnetworks:
- https://www.googleapis.com/compute/v1/projects/gcp-sharedvpc-host-project/regions/us-west2/subnetworks/private-us-west2
x_gcloud_bgp_routing_mode: GLOBAL
x_gcloud_subnet_mode: CUSTOM

Actual Behavior

no workie. seems that there is an assumption that (vpc) name will be project-local name, not a full path:

this'll do it every time: GET /compute/v1/projects/gcp-sharedvpc-service-project/global/networks/projects%2Fgcp-sharedvpc-host-project%2Fglobal%2Fnetworks%2Fshared-vpc

Steps to Reproduce

whip up a couple projects, share a VPC from the "host" project to the "service" project
from the service project, try to get some details about the shared VPC.

Important Factoids

"gcp-sharedvpc-host-project" has a VPC "shared-vpc", which is shared with "gcp-sharedvpc-service-project"

venkykuberan commented 4 years ago

If you want to pull the network details of host_project, you have to use host-project-id in data source where as rest of the config use your service-project-id

data "google_compute_network" "shared_vpc" {
  name    = <sharedvpc_host_network>
  project = <sharedvpc_host_project_id> 
}

lwatterworth-cldr commented 3 years ago

@venkykuberan adding the project in there worked great -- but I do have "Owner" in both projects, which may sway results?

If this is indeed how things should work, and the problem was PEBKAC, could we spin this bug report into a documentation request? Add a line or two, explicitly calling this out, maybe include shared VPC as an example?

thanks for the support!

rileykarson commented 3 years ago

Hey @lwatterworth-cldr!

You raise an interesting point about the difference between gcloud and Terraform with

While working in the service project, expecting to reference the shared VPC by it's full path name, as you can do with the gcloud CLI.

In Terraform (well, in the Google provider(s) at least) we've decomposed the full path/name of a resource into parts. So while gcloud expresses the network as projects/{{project}}/global/networks/{{name}}, in Terraform you'd express the same thing by filling in the project and name fields with the appropriate values. I'm not sure how we can make this change in expectations clear- doing it on every resource page is likely too heavy-handed, for example. Let me know if you have any ideas!

We do accept a path on some name/reference fields, and parse the project out of there. We could also consider doing that more consistently. It presents a bit of a problem- if a user also specifies project, there are 3 potential places the project is specified (project, resource, name/ref field) and we may need to use an unintuitive ordering for backwards compatibility reasons.

jimsnab commented 2 years ago

@rileykarson

When I specify the shared vpc as the network, subnetwork, it comes down to this conflict you mention.

resource google_container_cluster "my_cluster" {
  project = "my-service-project"
  network = "projects/my-shared-vpc-project/us-central1/networks/my-vpc"
  subnetwork = "projects/my-shared-vpc-project/us-central1/subnetworks/my-subnet"
  ...

Result:

Error: googleapi: Error 404: Not found: project "my-service-project" does not have a subnetwork named "my-subnet" in region "us-central1"., notFound

Note the project id mixup.

Is my issue this bug, or should I file a new one?

rileykarson commented 2 years ago

A new one, thanks @jimsnab!

davidlbyrne commented 7 months ago

I think there needs to be a way to detect if a project is a service project or a host project by dat source. Or this should be an attribute of the project resource.

rileykarson commented 7 months ago

It should be added as an attribute of project's resource and datasource for sure- https://cloud.google.com/compute/docs/reference/rest/v1/projects/get contains xpnProjectStatus. I'd suggest filing that separately as an enhancement request through https://github.com/hashicorp/terraform-provider-google/issues/new?assignees=&labels=enhancement&projects=&template=01_enhancement.yml

hashicorp / terraform-provider-google