Add support for pd-balanced storage option for Dataproc master and workers

[Bartosz-Stanisz]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

According to Dataproc docs it is possible to provision master and worker VMs with pd-balanced disk via gcloud and API. It would be nice to be able to use such config via Terraform - currently only option allowed are pd-standard and pd-ssd.

New or Affected Resource(s)

• google_dataproc_cluster

Potential Terraform Configuration

resource "google_dataproc_cluster" "cluster" {
  cluster_config {
    master_config {
      disk_config {
        boot_disk_type = "pd-balanced"
      }
    }

    worker_config {
      disk_config {
        boot_disk_type = "pd-balanced"
      }
    }

    preemptible_worker_config {
      disk_config {
        boot_disk_type = "pd-balanced"
      }
    }
  }
}

References

• Dataproc pd-ssd docs: https://cloud.google.com/dataproc/docs/concepts/compute/dataproc-pd-ssd

b/338675069

[melinath]

This restriction doesn't seem to actually be enforced on the Terraform side, so this is just a docs change.

[melinath]

That is, users should be able to just use the pd-balanced value today; the docs are just outdated.