Using Config Connector for referencing existing cloud resources

[eyalzek]

Describe the feature or resource

Basically, I would like to create a resource that only fetches a reference to an existing resource (in terraform terms this would be a data source as opposed to resource), e.g:

---
apiVersion: sql.cnrm.cloud.google.com/v1beta1
kind: SQLInstance
metadata:
  annotations:
    cnrm.cloud.google.com/project-id: ${GCP_PROJECT}
  name: mysql-instance

and this will look for an existing resource (as it does now), but instead of creating it if it doesn't exist or updating it's spec, just keep a reference of it's status and possibly spec.

Importance

This feature would be a nice-to-have. One of the benefits of using config connector, is that in certain cases you get access to attributes of a the cloud resource you created and this saves you the trouble of "hardcoding" those values. An example would be creating a SQLInstance and using composable to create an endpoint + svc to point to this instance. However, sometimes we prefer provisioning these resources with other tools (namely terraform) since it fits our use case and workflow much better. We can still create an endpoint and "hardcode" the IP of the instance (again, staying in the sql instance scenario), but it would have nice to get that benefit of referencing a resource without necessarily managing it with config connector. In terms of implementation, I can see this functionality being trigger by an annotation, for example cnrm.cloud.google.com/management-policy: reference/manage or something similar.

[xiaobaitusi]

Hi @eyalzek, we have heard this feature request from other channels as well and have added it into our backlog. We will let you know when we have more information.

Thanks for opening this enhancement issue!

[snuggie12]

I'd also like to add another benefit of this is certain chicken and egg resources can't be created by config connector such as the k8s cluster itself or the network underneath it but several resources have refs to networks so a read-only config connector resource would be great.

[bluemalkin]

This would be hugely handy. Really need something similar to Terraform's data sources. Especially since some name references must be the full ID (rather than the display name).

[sf-vorlov]

Any update on this feature request?

[sanketnadkarni]

probably you can import /export the resources into the KCC by using the binary config-connector, found here

further details on importing / exporting a single resource can be found here

there are options to bulk import / export also.

[snuggie12]

Any updates on this? Does server-side apply or the state into spec annotation solve this at all?

[diviner524]

We've had some previous discussions about this, and it's possible that we'll introduce a read-only mode for Config Connector. In this mode, Config Connector would solely be used to retrieve existing resources without making any updates.

There isn't a set timeline for implementing this feature at the moment.

[snuggie12]

We've had some previous discussions about this, and it's possible that we'll introduce a read-only mode for Config Connector. In this mode, Config Connector would solely be used to retrieve existing resources without making any updates.

There isn't a set timeline for implementing this feature at the moment.

@diviner524 the whole config connector is read-only or a resource is able to be read-only? For example, we have a config connector in one cluster creating a Project but then in another cluster we want the Project to be read-only so it can be used to reference for another type of resource (let's say a SQLInstance). We would want config-connector to not be read-only so it can create the SQLInstance but we do want the Project resource to be read-only. Does this line up with your current thinking?

[diviner524]

I am not actively working on the Config Connector project now, so not sure if this is currently being worked on.

@justinsb and @cheftako should be able to share more thoughts on the design.

[yinzara]

This is definitely something very important for us as well. Without this we can't migrate from custom helm operators that do similar things to this helm operator.