GoogleCloudPlatform / k8s-config-connector

GCP Config Connector, a Kubernetes add-on for managing GCP resources
https://cloud.google.com/config-connector/docs/overview
Apache License 2.0
892 stars 220 forks source link

Add support for BQ slot reservation management #586

Open tedelwartowski-bestbuy opened 2 years ago

tedelwartowski-bestbuy commented 2 years ago

Looking to have BQ slot reservation management added to config connector. The management of slot commitments is not a requirement for this use case, rather we would like the ability to create/update/delete slot reservations for use with existing slot commitments. Currently we are managing slot reservations manually and in some cases on a per project basis; it would be helpful if reservations and project assignment to a reservation could be managed in config connector.

Minimal Requirements:

diviner524 commented 2 years ago

@tedelwartowski-bestbuy Thank you for your suggestion! This resource (BigQueryReservation) is already in the list of resources we're planning to work on. I have also shared your request details in the internal tracking ticket.

We will let you know when we have more information.

tedelwartowski-bestbuy commented 1 year ago

@diviner524 - Is there any update on this request? Your comment made it sound like this was already planned however it has been over a year and we have not seen any support for slot reservation management in config connector.

tedelwartowski-bestbuy commented 1 year ago

Additionally, I would like to point out that BQ slot reservation management now includes autoscaling configurations: https://cloud.google.com/bigquery/docs/slots-autoscaling-intro. It would be good if this is also supported once autoscaling is GA.

diviner524 commented 1 year ago

Hi @tedelwartowski-bestbuy, yes we are working on supporting a list of new resources (including BigQueryReservation). The current ETA is late Q1 - early Q2, so we should be able to give an update very soon.

tedelwartowski-bestbuy commented 1 year ago

Hello, I noticed an Alpha CRD is not available for BigQueryReservationReservation (https://github.com/GoogleCloudPlatform/k8s-config-connector/blob/master/crds/bigqueryreservation_v1alpha1_bigqueryreservationreservation.yaml); however, there is no CRD available for creating assignments to a reservation (https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/bigquery_reservation_assignment). Do you have a status on when reservation assignments will be added? I also noticed, a CRD for BQ commitments (https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/bigquery_capacity_commitment) is also missing; this is not a priority for us, though it would be nice to have a ref to our commits in config.

diviner524 commented 1 year ago

@tedelwartowski-bestbuy We have released a v1alpha1 CRD BigQueryReservationCapacityCommitment in the latest version 1.108.0.

https://github.com/GoogleCloudPlatform/k8s-config-connector/blob/v1.108.0/crds/bigqueryreservation_v1alpha1_bigqueryreservationcapacitycommitment.yaml