advice for multi-cloud best practices

[Maldris]

Hi, I've used the blank template as this isn't so much an issue or feature request as a discussion to frame if a feature request is needed.

I'm working in an environment where we routinely need to instantiate cloud service connections both within the existing cloud as well as between clouds (i.e. running in aws, needing to write to azure).

the main scenario that I'm running into while investigating go-cloud is that we often have to instantiate that connection with details that aren't available using the shared credential strategy this library appears to use on its default entry points. (i.e. needing to assume a role in AWS or loading the relevant azure connection details from a secret and instantiating the connection using those details)

Given this, I'm wondering, would the advised best practice be for us to wrap go-cloud and use the cloud specific initialisers (i.e. gcpblob.OpenBucket) using our own credential loaders and acting as a wrapped entry point to the generic types. Or would there be scope for extending the existing behaviour to be able to retrieve more details about the session/credentials form the URL, or potentially from the context provided to the request (i.e. ctx.Value(k)), which would allow our domain specific content to only consist of establishing a session object for the respective cloud with the appropriate role/credentials.

If the latter would be preferred I'll open a separate feature request and finalise some of the design considerations in discussion before forking and creating a PR.

Thank you for your time and feedback.

[vangent]

wrap go-cloud and use the cloud specific initialisers (i.e. gcpblob.OpenBucket) using our own credential loaders and acting as a wrapped entry point to the generic types

Yes. If the URL openers don't work for your use case, the provider-specific constructors are intended to be maximally flexible

I am not sure exactly what you mean by "wrap go-cloud". You shouldn't have to wrap anything -- your initialization code can use the provider-specific constructors, which will return Go CDK concrete types (e.g., blob.Bucket), which you can then use normally.

would there be scope for extending the existing behaviour to be able to retrieve more details about the session/credentials form the URL, or potentially from the context provided to the request

I wouldn't want to retrieve details from the context.

If there are specific cases where we could add something to the URL to make it more usable, that is feasible. But we're not going to add anything that should be a secret to URLs (e.g., a path to a credentials file is potentially OK; the actual credentials is not).

If your code is constructing the URL, then you're probably better off using the explicit constructor. The URLs are intended to be used as inputs to the program being run, e.g., in a config file or command-line argument.

HTH

[Maldris]

Thanks @vangent, that matches what I was expecting, but as I know others often look in issues for similar matters I wanted to open the issue for future reference.

Regarding wrapping, we take comparable URL schemes as input, and have a session provider that instantiates sessions based on the type of cloud and the relevant permission and role scope. So we'd be creating a new equivalent to blob.OpenBucket that would inspect the URL, and call the relevant platform specific initializer using a session from our provider. We considered abstracting this somewhat if we were to instead attach a session to context, or identify the secret path in the URL for retrieval via the secrets package (though that potentially creates a call recursion issue, in addition to an import loop, so a far from ideal scenario), but we all agreed this would be somewhat of a kludge and was intended as a fallback only.

We'll create wrappers as discussed above and use the existing session provider we've been using for the small number of systems we've needed it for previously, as we do this, we'll see if the strategy becomes suitably generic to either make a separate helper repository ourselves, or bring any learnings back that may be useful here.

Thanks for your input and time.