GKE Nodepool Custom Image VPC

[chrislovecnm]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

Add the capability to define a custom image name in TF. When defining a node_config add the capability to define an image name.

Add the capability to define the VPC that a google image lives in for a GKE cluster. When defining a node_config you have able to define an image_type. An image may live in a different vpc, from where the GKE cluster resides.

New or Affected Resource(s)

• google_container_cluster

Potential Terraform Configuration

resource "google_container_node_pool" "primary_preemptible_nodes" {
  name       = "my-node-pool"
  location   = "us-central1"
  cluster    = "${google_container_cluster.primary.name}"
  node_count = 1

  node_config {
    image_type = "CUSTOM"
    image = "my-image"
    image_vpc = "my-vpc"

    preemptible  = true
    machine_type = "n1-standard-1"

    metadata {
      disable-legacy-endpoints = "true"
    }

    oauth_scopes = [
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
    ]
  }
}

References

This is a valid command in gcloud

gcloud container node-pools create test-cos --cluster=test-infra \
--image-type=CUSTOM --image=cos-test \
--image-project=cos-gke-test --machine-type=custom-96-230400 \
--node-version=1.11.6-gke.2 --num-nodes=2 --zone=us-central1-c
--project=test --min-cpu-platform=skylake

The above command will create a node pool using the cos-gke-test machine image. This image lives in the cos-gke-test project, while the gke cluster lives in the test project.

b/351038278

[emilymye]

May I ask where you saw this gcloud command? I can't find documentation for this feature, so a link to the docs, API reference, or similar would be helpful.

I also just updated my gcloud and I get this output from running gcloud beta container node-pools create (GA returns subset of these flags):

gcloud beta container node-pools create NAME [optional flags]
  optional flags may be  --accelerator | --cluster | --disk-size | --disk-type |
                         --enable-autorepair | --enable-autoscaling |
                         --enable-autoupgrade | --enable-cloud-endpoints |
                         --help | --image-type | --local-ssd-count |
                         --machine-type | --max-nodes | --max-pods-per-node |
                         --metadata | --metadata-from-file |
                         --min-cpu-platform | --min-nodes | --node-labels |
                         --node-taints | --node-version | --num-nodes |
                         --preemptible | --region | --scopes |
                         --service-account | --tags |
                         --workload-metadata-from-node | --zone

which doesn't include image or image-project. Do you mind running this gcloud command with --log-http to show what it's calling?

[chrislovecnm]

Yah once I have a service account I will. Super secret ninja magic. The command was provided to me via google support.

[eyalzek]

@chrislovecnm any update on this?

[roaks3]

It looks like there might be an existing undocumented field in the API called nodeImageConfig under nodeConfig that supports these image fields when imageType is CUSTOM. I can't quite tell which API(s) support this field, but my best guess is that it is available and could be added to TF.