search

GoogleCloudPlatform / k8s-config-connector

GCP Config Connector, a Kubernetes add-on for managing GCP resources
https://cloud.google.com/config-connector/docs/overview
Apache License 2.0
893 stars 222 forks source link

Configure Cloud Build Worker Pools #522

Open flunderpero opened 3 years ago

flunderpero commented 3 years ago

Please add Cloud Build Worker Pools (https://cloud.google.com/build/docs/private-pools/worker-pool-config-file-schema)

maqiuyujoyce commented 3 years ago

Hi @flunderpero , thank you for your feedback. We've added it to the list of resources we're looking into. Meanwhile, could you share some information about how important this resource is? Is it a blocker, a friction point, or a nice-to-have?

flunderpero commented 3 years ago

It's a friction point. We are desperate to use the actual feature (worker pools) so we configure the resource using a (Python) script. We do this for all that's missing in config connector (which is sadly quite a lot, but it's getting slowly better.)

Are there plans to release the CRD along with the APIs when a new feature is added to Google Cloud? Or will config connector always be an afterthought?

maqiuyujoyce @.***> schrieb am Di., 3. Aug. 2021, 21:30:

Hi @flunderpero https://github.com/flunderpero , thank you for your feedback. We've added it to the list of resources we're looking into. Meanwhile, could you share some information about how important this resource is? Is it a blocker, a friction point, or a nice-to-have?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/k8s-config-connector/issues/522#issuecomment-892107096, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADIDYHJSLEFMR7ZBQTZBU63T3A7U7ANCNFSM5BJ3WWJQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

maqiuyujoyce commented 3 years ago

Sorry to hear that you have to do workaround for many missing resources in Config Connector, @flunderpero . We'll keep you posted when we have any updates on this resource.

Are there plans to release the CRD along with the APIs when a new feature is added to Google Cloud? Or will config connector always be an afterthought?

Yes we are working on it. In the short term, most resources will be supported in Config Connector after the GCP APIs release them. Good news is that there are already a few API teams starting working with us at an earlier stage of their API design/implementation. And Config Connector integration of those APIs have been/will be released at the same time when the API itself becomes available. So hopefully it will get better over time.

flunderpero commented 3 years ago

I really appreciate your effort. It was such a relieve to convert our (rather large) GCP resource setup/management code to a declarative system in config connector. Now Everytime we have to work around that, it hurts that much more. :-)

maqiuyujoyce @.***> schrieb am Mi., 4. Aug. 2021, 23:35:

Sorry to hear that you have to do workaround for many missing resources in Config Connector, @flunderpero https://github.com/flunderpero . We'll keep you posted when we have any updates on this resource.

Are there plans to release the CRD along with the APIs when a new feature is added to Google Cloud? Or will config connector always be an afterthought?

In the short term, the answer is yes. Good news is that there are already a few API teams starting working with us at an earlier stage of their API design/implementation. And Config Connector integration of those APIs have been/will be released at the same time when the API itself becomes available. So hopefully it will get better over time.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/k8s-config-connector/issues/522#issuecomment-892990542, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADIDYHOEB4DIFZNSKRMIO43T3GXATANCNFSM5BJ3WWJQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

OperationalFallacy commented 1 year ago

I have the same question as the author of this issue. It is a blocker, we can't run use cloud build in the enterprise environments without worker pools.

Since its been a few years after the last update, I'd like to check if there are still plans to implement it?

diviner524 commented 1 year ago

@maqiuyujoyce Do you know if this resource can be auto-generated as a v1alpha1 CRD?

https://registry.terraform.io/providers/hashicorp/google-beta/4.83.0/docs/resources/cloudbuild_worker_pool

Furthermore, @OperationalFallacy we've recently added a guidance on how to add a new Config Connector CRD using the Terraform Provider, and this guidance should be applicable to this particular resource. We're open to contributions and would be more than happy to assist with reviewing any changes you'd like to make.

https://github.com/GoogleCloudPlatform/k8s-config-connector/blob/master/README.NewResourceFromTerraform.md

OperationalFallacy commented 1 year ago

@diviner524 Thanks for the suggestion; given the number of steps involved (2 primary and 12 substeps - 14 total if ChatGPT is not mistaken), I prefer to concentrate on the business value that the service delivers to the customers rather than trying to make the service work with all these layered abstractions and third-party vendors.

How about counter offer: your team implements it and we help with testing in real enterprise settings .

maqiuyujoyce commented 1 year ago

@diviner524 Yes we should be able to generate them as the resource has been supported in TF v4.

erik-carlson commented 9 months ago

@maqiuyujoyce there have been two releases since the last comment and I don't see the auto-generated resource created. Is there something that needs to happen to pull that in? We would be interested in validating the alpha resource.

maqiuyujoyce commented 9 months ago

Hi @erik-carlson , thank you for the follow up! The auto-generated resources can be automatically generated, but with a manual trigger. At the moment, we don't automatically generate new resources regularly. That's why CloudBuildWorkerPool is not yet supported. Are you willing to do the auto-generation of the resource if we provide the instructions?

@diviner524 it looks like we didn't make the instructions about resource autogen public. Is there any concern about it?

Also, @erik-carlson , if you'd like to CloudBuildWorkerPool resource to be supported in Config Connector, could you file a resource request via Cloud support?

erik-carlson commented 9 months ago

Maybe, depending on the effort? We created a support case last month, with the result being this issue: https://issuetracker.google.com/issues/318546227

maqiuyujoyce commented 9 months ago

Hi @erik-carlson , sorry for the delayed response.

I double checked the autogen instructions and realized that the doc is outdated. While we are still working on prioritizing the doc cleanup, are you open to support it through our standard process: https://github.com/GoogleCloudPlatform/k8s-config-connector/blob/master/README.NewResourceFromTerraform.md ?