Add support for PSC in google_dataproc_metastore_service

[brokenjacobs]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

google_dataproc_metastore_service now supports private service connect to connect to a VPC without peering. This support just entered pre-GA and is documented here: https://cloud.google.com/dataproc-metastore/docs/psc-ilb

This is a great feature because otherwise dataproc metastore service provides no way to configure IP address ranges, and constantly configures ranges that conflict with on-premise networks. This leads to very bad terraform...

New or Affected Resource(s)

• google_dataproc_metastore_service

Potential Terraform Configuration

Assuming the appropriate IAM:

# Propose what you think the configuration to take advantage of this feature should look like.
# We may not use it verbatim, but it's helpful in understanding your intent.
resource "google_project_service" "servicenetworking" {
  service = "servicenetworking.googleapis.com"
  disable_on_destroy = false
}

resource "google_compute_network" "network" {
  name                    = "my-network"
  auto_create_subnetworks = false
  depends_on = [google_project_service.servicenetworking]
}

resource "google_compute_subnetwork" "subnet" {
  name          = "metastore-subnet"
  ip_cidr_range = "10.2.0.0/16"
  region        = "us-central1"
  network       = google_compute_network.network.id
}

resource "google_dataproc_metastore_service" "default" {
  provider   = google-beta
  service_id = "metastore-srv"
  location   = "us-central1"
  port       = 9080
  tier       = "DEVELOPER"

  maintenance_window {
    hour_of_day = 2
    day_of_week = "SUNDAY"
  }

  hive_metastore_config {
    version = "2.3.6"
  }
  network_config {
    subnetwork = google_compute_subnetwork.subnet.id
  }
}

References

• Internal issue: b/207405119

[brokenjacobs]

Looks like my sample is wrong, this uses a loadbalancers ILB and doesn't require a peering subnetwork. --consumer-subnetworks is the gcloud cli option.

[DrFaust92]

Ill take a look at this

[zli82016]

@brokenjacobs , do you have another sample? Thanks.

[brokenjacobs]

I mean I could make something up, but wouldn’t this depend on the api?

[zli82016]

I talked with API team and I am good, thanks.

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.