Tags and labels configured via Terraform for google_notebooks_runtime, not visible in Google UI

[rlopezl]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v1.2.9 on linux_amd64

Affected Resource(s)

• google_notebook_runtiume

Terraform Configuration Files

* main.tf
locals {
  prefix = join("-", tolist(["go", lower(var.labels.region), var.labels.enterprise, var.labels.account, var.labels.system, ""]))
}

resource "google_notebooks_runtime" "runtime" {
  count = var.create_workbench ? 1 : 0

  name = format(
    "${local.prefix}%s-notebook%.2d-%s",
    var.notebook_name,
    var.offset,
    var.labels.environment
  )

  project  = var.project_id
  location = var.location

  dynamic "access_config" {
    for_each = (
      var.access_config == null ? [] : [true]
    )
    content {
      access_type   = var.access_config.access_type
      runtime_owner = var.access_config.runtime_owner
    }
  }

  dynamic "software_config" {
    for_each = (
      var.software == null ? [] : [true]
    )
    content {
      notebook_upgrade_schedule    = var.software.notebook_upgrade_schedule
      enable_health_monitoring     = var.software.enable_health_monitoring
      idle_shutdown                = var.software.idle_shutdown
      idle_shutdown_timeout        = var.software.idle_shutdown_timeout
      install_gpu_driver           = var.software.install_gpu_driver
      custom_gpu_driver_path       = var.software.custom_gpu_driver_path
      post_startup_script          = var.software.post_startup_script
      post_startup_script_behavior = var.software.post_startup_script_behavior

      dynamic "kernels" {
        for_each = (
          var.software.kernels == null ? [] : [true]
        )
        content {
          repository = var.software.kernels.repository
          tag        = var.software.kernels.tag
        }
      }
    }
  }

  virtual_machine {
    virtual_machine_config {
      machine_type = var.machine_type

      dynamic "data_disk" {
        for_each = var.data_disk
        iterator = disk

        content {
          interface = disk.value.interface
          mode      = disk.value.mode
          source    = disk.value.source
          type      = disk.value.type

          initialize_params {
            description  = disk.value.description
            disk_name    = disk.value.disk_name
            disk_size_gb = disk.value.disk_size_gb
            disk_type    = disk.value.disk_type
          }
        }
      }

      dynamic "container_images" {
        for_each = (
          var.container_repository == null ? [] : [true]
        )
        content {
          repository = var.container_repository
          tag        = var.container_tag
        }
      }

      dynamic "encryption_config" {
        for_each = (
          var.kms_key == null ? [] : [true]
        )
        content {
          kms_key = var.kms_key
        }
      }

      dynamic "shielded_instance_config" {
        for_each = (
          var.enable_secure_boot == null
          && var.enable_vtpm == null
          && var.enable_integrity_monitoring == null
        ) ? [] : [true]
        content {
          enable_secure_boot          = var.enable_secure_boot
          enable_vtpm                 = var.enable_vtpm
          enable_integrity_monitoring = var.enable_integrity_monitoring
        }
      }

      dynamic "accelerator_config" {
        for_each = (
          var.accelerator_core_count == null ? [] : [true]
        )
        content {
          core_count = var.accelerator_core_count
          type       = var.accelerator_type
        }
      }

      network           = var.network
      subnet            = var.subnet
      internal_ip_only  = var.internal_ip_only
      tags              = var.tags
      metadata          = var.metadata
      nic_type          = var.nic_type
      reserved_ip_range = var.reserved_ip_range

      labels = merge(
        var.labels,
        {
          "resource_name" = format(
            "${local.prefix}%s-notebook%.2d-%s",
            var.notebook_name,
            var.offset,
            var.labels.environment
          ),
          "resource_type" = "notebook",
        }
      )
    }
  }
}

* terraform.tfvars

internal_ip_only = true
  machine_type     = "n1-standard-1"
  notebook_name    = "dummy"
  project_id       = var.project_id
  location         = var.region
  network          = "projects/go-st-bcn-lacld-p4013-pro/global/networks/go-euw4-st-bcn-lacld-spoke-net01-pro"
  subnet           = "projects/go-st-bcn-lacld-p4013-pro/regions/europe-west4/subnetworks/go-euw4-st-bcn-lacld-spoke-1-pro"

  access_config = {
    access_type   = "SINGLE_USER"
    runtime_owner = "rlopez@st.com"
  }

  data_disk = [
    {
      disk_size_gb = "50"
      disk_type    = "PD_STANDARD"
    }
  ]

  software = {
    idle_shutdown         = true
    idle_shutdown_timeout = 240
  }

  labels = {
  "provider"                = "go",
  "region"                  = "euw4",
  "enterprise"              = "cxb",
  "account"                 = "poc",
  "system"                  = "tst"
  "environment"             = "poc",
  "cmdb_name"               = "",
  "security_exposure_level" = "mz",
  "status"                  = "",
  "on_service"              = "yes",
}

Expected Behavior

When we'll see the details of the notebook runtime/instance, we suppose to see the configured labels.

Actual Behavior

When we query the notebook instance through google-sdk, the tags are visible:

But, when we trying to visualize the labels through the Google Portal, none of them are enable:

Steps to Reproduce

1. terraform apply
2. gcloud notebooks runtimes describe --location=europe-west4 notebooks-runtime

b/315120616

[edwardmedia]

@rlopezl Can you share the debug log?

Did you provide the var.tags from a different way? ( I did not see it in the terraform.tfvars )

[edwardmedia]

@rlopezl using below config, I can build a runtime which has both tags and labels in the following section. Can you try this config to see what you can get?

resource "google_notebooks_runtime" "runtime" {
  name     = "issue16634"
  location = "us-central1"
  access_config {
    access_type   = "SINGLE_USER"
    runtime_owner = "admin@hashicorptest.com"
  }
  virtual_machine {
    virtual_machine_config {
      machine_type = "n1-standard-4"
      data_disk {
        initialize_params {
          disk_size_gb = "100"
          disk_type    = "PD_STANDARD"
        }
      }
      tags = ["foo", "bar"]
      labels = {
        environment = "dev"
      }
    }
  }
}

virtualMachine:
  virtualMachineConfig:
    acceleratorConfig: {}
    bootImage: {}
    dataDisk:
      initializeParams:
        diskSizeGb: '100'
        diskType: PD_STANDARD
    labels:
      environment: dev
    machineType: n1-standard-4
    tags:
    - foo
    - bar
    zone: us-central1-c

[edwardmedia]

@rlopezl is this still an issue?

[rlopezl]

Yes, there are visible through google sdk, as I posted in the first comment, but they are not visible if you check it out in Google Cloud Portal.

* vars.tfvars

variable "create_workbench" {
  description = "Controls if Vertex Workbench should be created"
  type        = bool
  default     = true
}

variable "notebook_name" {
  description = "The name specified for the Notebook runtime"
  type        = string
  default     = ""
}

variable "location" {
  description = "A reference to the zone where the machine resides"
  type        = string
  default     = "europe-west4"
}

variable "project_id" {
  description = "The ID of the project in which the resource belongs"
  type        = string
}

variable "offset" {
  description = "The offset to be added to the network counter"
  type        = number
  default     = 1
}

variable "machine_type" {
  description = "The Compute Engine machine type used for runtimes"
  type        = string
}

variable "data_disk" {
  description = <<EOH
    List of subnets being created.
    <a name=description:></a>[description:](#description:) An optional description of this resource. Provide this property when you create the resource.
    <a name=disk_name:></a>[disk_name:](#disk_name:) Specifies the disk name
    <a name=disk_size_gb:></a>[disk_size_gb:](#disk_size_gb:) Specifies the size of the disk in base-2 GB
    <a name=disk_type:></a>[disk_type:](#disk_type:) The type of the boot disk attached to this runtime
    <a name=interface:></a>[iam_users:](#interface:) Specifies the disk interface to use for attaching this disk
    <a name=mode:></a>[mode:](#mode:) The mode in which to attach this disk, either READ_WRITE or READ_ONLY
    <a name=source:></a>[source:](#source:) Specifies a valid partial or full URL to an existing Persistent Disk resource
    <a name=type:></a>[type:](#type:) Specifies the type of the disk, either SCRATCH or PERSISTENT
    EOH
  type = list(object({
    description  = optional(string)
    disk_name    = optional(string)
    disk_size_gb = optional(string)
    disk_type    = optional(string)
    interface    = optional(string)
    mode         = optional(string)
    source       = optional(string)
    type         = optional(string)
  }))
  default = []
}

variable "container_repository" {
  description = "The path to the container image repository."
  type        = string
  default     = null
}

variable "container_tag" {
  description = "The tag of the container image. If not specified, this defaults to the latest tag"
  type        = string
  default     = null
}

variable "kms_key" {
  description = "The Cloud KMS resource identifier of the customer-managed encryption key used to protect a resource"
  type        = string
  default     = null
}

variable "enable_secure_boot" {
  description = "Defines whether the instance has Secure Boot enabled"
  type        = bool
  default     = true
}

variable "enable_vtpm" {
  description = "Defines whether the instance has the vTPM enabled"
  type        = bool
  default     = true
}

variable "enable_integrity_monitoring" {
  description = "Defines whether the instance has integrity monitoring enabled"
  type        = bool
  default     = true
}

variable "accelerator_core_count" {
  description = "The type of accelerator model"
  type        = string
  default     = null
}

variable "accelerator_type" {
  description = "The count of cores of this accelerator"
  type        = number
  default     = null
}

variable "network" {
  description = "The Compute Engine network to be used for machine communications"
  type        = string
  default     = null
}

variable "subnet" {
  description = "The Compute Engine subnetwork to be used for machine communications"
  type        = string
  default     = null
}

variable "internal_ip_only" {
  description = "Defines whether runtime will only have internal IP addresses"
  type        = bool
  default     = true
}

variable "tags" {
  description = "The Compute Engine tags to add to runtime"
  type        = list(string)
  default     = []
}

variable "metadata" {
  description = "The Compute Engine metadata entries to add to virtual machine"
  type        = map(string)
  default     = {}
}

variable "nic_type" {
  description = "The type of vNIC to be used on this interface. This may be gVNIC or VirtioNet. Possible values are UNSPECIFIED_NIC_TYPE, VIRTIO_NET, and GVNIC"
  type        = string
  default     = null
}

variable "reserved_ip_range" {
  description = "The reserved IP Range name is used for VPC Peering"
  type        = string
  default     = null
}

variable "software" {
  description = <<EOH
  A configurable object that defines the details of the software.
  <a name=notebook_upgrade_schedule:></a>[notebook_upgrade_schedule:](#notebook_upgrade_schedule:) Cron expression in UTC timezone for schedule instance auto upgrade
  <a name=enable_health_monitoring:></a>[enable_health_monitoring:](#enable_health_monitoring:) Verifies core internal services are running
  <a name=idle_shutdown:></a>[idle_shutdown:](#idle_shutdown:) Defines whether runtime will automatically shutdown after
  <a name=idle_shutdown_timeout:></a>[idle_shutdown_timeout:](#idle_shutdown_timeout:) Time in minutes to wait before shuting down runtime
  <a name=install_gpu_driver:></a>[install_gpu_driver:](#install_gpu_driver:)  Defines whethe install Nvidia Driver automatically
  <a name=custom_gpu_driver_path:></a>[custom_gpu_driver_path:](#custom_gpu_driver_path:) Specify a custom Cloud Storage path where the GPU driver is stored
  <a name=post_startup_script:></a>[post_startup_script:](#post_startup_script:) Path to a Bash script that automatically runs after a notebook instance fully boots up
  <a name=post_startup_script_behavior:></a>[post_startup_script_behavior:](#post_startup_script_behavior:) Behavior for the post startup script. Possible values are POST_STARTUP_SCRIPT_BEHAVIOR_UNSPECIFIED, RUN_EVERY_START, and DOWNLOAD_AND_RUN_EVERY_START
  <a name=kernels.repository:></a>[kernels.repository:](#kernels.repository:) The path to the container image repository
  <a name=kernels.tag:></a>[kernels.tag:](#kernels.tag:) The tag of the container image
  EOH
  type = object({
    notebook_upgrade_schedule    = optional(string)
    enable_health_monitoring     = optional(bool)
    idle_shutdown                = optional(bool)
    idle_shutdown_timeout        = optional(number)
    install_gpu_driver           = optional(bool)
    custom_gpu_driver_path       = optional(string)
    post_startup_script          = optional(string)
    post_startup_script_behavior = optional(string)
    kernels = optional(object({
      repository = optional(string)
      tag        = optional(string)
    }))
  })
  default = null
}

variable "access_config" {
  description = <<EOH
  Specifies the login configuration for Runtime.
  <a name=access_type:></a>[access_type:](#access_type:) The type of access mode this instance
  <a name=runtime_owner:></a>[runtime_owner:](#runtime_owner:) The owner of this runtime after creation
  EOH
  type = object({
    access_type   = optional(string)
    runtime_owner = optional(string)
  })
  default = null
}

variable "labels" {
  description = "A mapping of labels to assign to all resources"
  type        = map(string)
}

[edwardmedia]

@rlopezl not quite sure what you meant here. Did you say you are able to see the tags & labels by running gcloud like the result here? If this is the case, why did you think the issue can be solved in the terraform provider?

Yes, there are visible through google sdk, as I posted in the first comment, but they are not visible if you check it out in Google Cloud Portal.

From the provider perspective, as long as the attributes are successfully set by the api. Whether they are visible or not, there could be other reasons. Does this make sense?

Are you able to create a runtime via gcloud whose tags and labels are visible? If yes, can you share its debug log? ( attaching --log-http in gcloud)

[alfonso2009]

@edwardmedia If you follow the next steps, you can see our requirement:

1. Apply with Terraform a Managed Notebook
2. You can see in GCP UI that the value of labels is None
3. If you call to the API (GET https://notebooks.googleapis.com/v1/${NOTEBOOK}), the labels are set in virtualMachine.VirtualMachineConfig.labels field:

{ "name": "${NOTEBOOK}", "virtualMachine": { "virtualMachineConfig": { "zone": "europe-west4-b", "machineType": "n1-standard-1", "dataDisk": { "initializeParams": { "diskSizeGb": "50", "diskType": "PD_STANDARD" } }, "shieldedInstanceConfig": { "enableSecureBoot": true, "enableVtpm": true, "enableIntegrityMonitoring": true }, "acceleratorConfig": {}, "internalIpOnly": true, "labels": { "on_service": "yes", "provider": "go", "environment": "poc" }, "bootImage": {} } }, "state": "INITIALIZING", "accessConfig": { "accessType": "SINGLE_USER", "runtimeOwner": "${EMAIL}", "proxyUri": "${PROXYURI}" }, "softwareConfig": { "enableHealthMonitoring": true, "idleShutdown": true, "idleShutdownTimeout": 240, "upgradeable": false, "version": "m112" }, "createTime": "2023-12-05T12:16:49.157641690Z", "updateTime": "2023-12-05T12:17:32.566948909Z", "migrated": false, "runtimeMigrationEligibility": { "warnings": [ "UNSUPPORTED_OS", "SINGLE_USER", "GOOGLE_MANAGED_NETWORK" ] } }

4. Call to API (PATCH https://notebooks.googleapis.com/v1/${NOTEBOOK}?updateMask=labels) to introduce the same labels. Then if we recall to the API to recover the notebook details, the labels are inside VirtualMachineConfig and at first level of the JSON reponse as new variable called labels and they are available in the GCP UI now. 

{ "name": "${NOTEBOOK}", "virtualMachine": { "virtualMachineConfig": { "zone": "europe-west4-b", "machineType": "n1-standard-1", "dataDisk": { "initializeParams": { "diskSizeGb": "50", "diskType": "PD_STANDARD" } }, "shieldedInstanceConfig": { "enableSecureBoot": true, "enableVtpm": true, "enableIntegrityMonitoring": true }, "acceleratorConfig": {}, "internalIpOnly": true, "labels": { "on_service": "yes", "provider": "go", "environment": "poc" }, "bootImage": {} } }, "state": "INITIALIZING", "accessConfig": { "accessType": "SINGLE_USER", "runtimeOwner": "${EMAIL}", "proxyUri": "${PROXYURI}" }, "softwareConfig": { "enableHealthMonitoring": true, "idleShutdown": true, "idleShutdownTimeout": 240, "upgradeable": false, "version": "m112" }, "createTime": "2023-12-05T12:16:49.157641690Z", "updateTime": "2023-12-05T12:27:21.280966072Z", "labels": { "on_service": "yes", "provider": "go", "environment": "poc" }, "migrated": false, "runtimeMigrationEligibility": { "warnings": [ "UNSUPPORTED_OS", "SINGLE_USER", "GOOGLE_MANAGED_NETWORK" ] } }

5. In the provider https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/notebooks_runtime there isn´t a variable called labels at top level of workbench to set.

This is the reason why we think that is a bug in the provider, because yo can set them terraform way, only for API.

Thanks!

[edwardmedia]

Oh I see the labels is available in the api. We can add it to the resource. Change the label to enhancement accordingly.

PS, I don't see Tags at the same level though

[bcreddy-gcp]

There are labels for various components of the runtime. Disks have their labels, Virtual machine has its label and the runtime itself has its label. The labels displayed in the UI are the runtime labels. They can be set with the following config:

resource "google_notebooks_runtime" "runtime" {
  name     = "issue16634"
  location = "us-central1"
  access_config {
    access_type   = "SINGLE_USER"
    runtime_owner = "admin@hashicorptest.com"
  }
  virtual_machine {
    virtual_machine_config {
      machine_type = "n1-standard-4"
      data_disk {
        initialize_params {
          disk_size_gb = "100"
          disk_type    = "PD_STANDARD"
        }
      }
      tags = ["foo", "bar"]
    }
  }
  labels = {
    environment = "dev"
  }
}

For the tags, they were never visible in the UI and is not a Terraform issue. If this is of interest, a feature request can be created.