googleapis / python-dataproc

This library has moved to https://github.com/googleapis/google-cloud-python/tree/main/packages/google-cloud-dataproc
Apache License 2.0
48 stars 30 forks source link

DataProc: Cluster config doesn't show lifecycle config for clusters created outside 'global' region #2

Closed jonescameron closed 4 years ago

jonescameron commented 5 years ago

Trying to write a function in Python that parses cluster config files, I've noticed that when I import dataproc_v1beta2, I can see lifecycle info in the cluster config with the format:

lifecycle_config {
  idle_delete_ttl {
    seconds: 21600
  }
}

which is perfect, however when getting config from clusters created in any region other than 'global', this information is not present.

Code to reproduce:

from google.cloud import dataproc_v1beta2
from google.cloud.dataproc_v1.gapic.transports import cluster_controller_grpc_transport as ccgt

project = 'myProject'
region = 'europe-west1'
transport = ccgt.ClusterControllerGrpcTransport(address='{}-dataproc.googleapis.com:443'.format(region))

client = dataproc_v1beta2.ClusterControllerClient(transport)
c1 = client.get_cluster(project, region, 'test-cluster-in-europe-sched')
c2 = client.get_cluster(project, region, 'test-cluster-in-europe-no-sched')

region = 'global'

client = dataproc_v1beta2.ClusterControllerClient()
c3 = client.get_cluster(project, region, 'test-cluster-with-schedule')
c4 = client.get_cluster(project, region, 'test-cluster-without-schedule')

c1cfg = c1.config
c2cfg = c2.config
c3cfg = c3.config
c4cfg = c4.config

When running print() on these 4 ClusterConfigs, lifecycle_config only appears on c3, where I would have expected it on c1 also.

Thanks for your time.

tseaver commented 5 years ago

Thanks for the report!

bradmiro commented 4 years ago

As of 0.7.0 this should be fixed for dataproc_v1beta2.