Automatically configure provider credentials when installing a Configuration

[negz]

A Configuration may include zero or more Compositions, and depend on zero or more Configurations (which in turn may include zero or more Compositions). Each Composition specifies one or more composed resources, which must each reference a "Provider" (of the credential variety per https://github.com/negz/example-cnp/issues/2). Providers are currently managed outside the lifecycle of Configurations; it's assumed that the platform operator would determine what Provider (credentials) needed to exist after installing their desired Configurations, and create them.

Ideally it would be possible for this credential configuration process to be semi-automated when a Configuration was installed via Upbound Cloud or the crossplane CLI tool; for the platform operator to be prompted to supply the appropriate credentials.

[negz]

I don't think we can assume that only one set of provider credentials needs to be configured. It's an edge case, but even a single composition could hypothetically compose resources that used different provider credentials. For example an EKS composition may want Crossplane to use one IAM user to manage the control plane and another to manage node pools. If the Composition referenced two different "providers", we'd need to ensure both of those providers existed (and pointed to a valid credential source such as a Secret).

[negz]

We may be able to simplify this problem by introducing the concept of a default Provider, which per https://github.com/negz/example-cnp/issues/2 are more like provider credentials:

• Managed resources without a providerRef would use the Provider annotated as the default.
• Configuration (package) authors would be encouraged to omit providerRef in their Compositions by convention.

This way in the happy path Crossplane need only to ensure one set of default Provider credentials exist per installed provider (i.e. one for AWS, one for Azure, etc) and most Configurations would work. Configurations could choose to be opinionated using a specific provider - if they did they would be outside the happy path and it would be up to the platform operator to manually create the necessary provider(s) for those configurations.

[negz]

It's possible we could even use a well known name rather than an annotation to signify the default provider credentials. i.e. if metadata.name == default the provider is the default provider. This way the problem of setting up credentials becomes:

• When installing a Provider (package) create provider credentials named default if they don't already exist.
• Install your configurations. If they're not opinionated about providers, they'll just use the one named default.

[hasheddan]

@negz this would be made even easier by the distinction of Configuration and Provider as packages. Still in favor of potentially changing the name of the Provider object type though

[muvaf]

One of the problems with including Provider object in a bundle is the deletion problem: Provider objects get deleted immediately if you delete the package, hence, managed resources do not get a chance to be deleted with the credentials. In the past, I've been on the side of not including it in the bundle and just referring them in the Sample Stack CR, though, we didn't take that route due to UbC not being able to create a Provider resource manually.

However, I still think that Provider object should be created by the user and its name should be given as input to the package if it needs to reference a Provider. Most of the platform I've interacted with works in a similar fashion; create the authentication object(could be just a simple Secret but in this case the auth is Secret+Provider) in a different menu/context and then refer to it whenever you'd like to use.

This way in the happy path Crossplane need only to ensure one set of default Provider credentials exist per installed provider (i.e. one for AWS, one for Azure, etc) and most Configurations would work. Configurations could choose to be opinionated using a specific provider - if they did they would be outside the happy path and it would be up to the platform operator to manually create the necessary provider(s) for those configurations.

@negz in case they don't want to use default Provider, will they be able to supply the name of the provider to be used by the Configuration?

[hasheddan]

One of the problems with including Provider object in a bundle is the deletion problem: Provider objects get deleted immediately if you delete the package, hence, managed resources do not get a chance to be deleted with the credentials.

If creating the Provider object is a side-effect of installing a Provider type package then we can also control how it gets cleaned up.

[muvaf]

If creating the Provider object is a side-effect of installing a Provider type package then we can also control how it gets cleaned up.

I was meaning Configuration type package. Right now, sample stacks create Provider objects and platform builders assign deletion priority of -1 to keep it around until all others are gone. If we are able to keep Provider object that refers to a Secret out of Configurations, then we would not need to worry about this problem. Though this would require supplying the name of Provider to Configuration. Well, actually you need to give name of the Secret even if Provider is included. The default Provider would cover only the cases where you have 1 Provider object and you're OK with not using more than 1 Provider object in that cluster.