data-dot-all / dataall

A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.
https://data-dot-all.github.io/dataall/
Apache License 2.0
232 stars 82 forks source link

Add Client Factory for Regionalized Endpoints #561

Open noah-paige opened 1 year ago

noah-paige commented 1 year ago

Is your idea related to a problem? Please describe. For api requests to AWS global services we sometimes need to point to the specific regional endpoint if we want communication to navigate via PrivateLink (e.g. STS). There is redundancy in defining this STS client in multiple areas of the codebase passing the specific regional endpoint to communicate with.

Describe the solution you'd like Create either a client factory to handle setting up the AWS service client to communicate with the regional endpoint for said services and/or use environment variables to use regional endpoints as default behavior.

P.S. Don't attach files. Please, prefer add code snippets directly in the message body.

dlpzx commented 1 year ago

Hi @noah-paige, I am validating this feature against the data.all V2 code in main-v2. Let's see, we have the following boto3 wrappers:

In base

In core