Open hsheth2 opened 2 years ago
This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io
Our new Redshift connector fully relies on Redshift Client and if I'm not mistaken if you should be able to use IAM credential (except for profiling which still relies on SqlAlchemy) This is from the above StackOverflow post:
conn = redshift_connector.connect(
iam=True,
database='dev',
db_user='<username>', # the database user in call to get-cluster-credentials
cluster_identifier='my-redshift-cluster', # identifier of your cluster
profile='redshift_profile' # profile in ~./aws/config with correct permissions
)
and this should be set by:
type: redshift
config:
extra_client_options:
iam: true
db_user: <username>
cluster_identifier: my-redshift-cluster
profile: redshift_profile
@treff7es that's pretty nifty - can you update the docs to show an example of that, and then we can close out this issue
We should be able take the user's IAM credentials and call
redshift:GetClusterCredentials
to get the credentials dynamically.As per https://stackoverflow.com/questions/44243169/connect-to-redshift-using-python-using-iam-role, the AWS-native redshift_connector driver has support for this built in, but it's possible to do ourselves as well.
We're currently using the Redshift sqlalchemy dialect with the
psycopg2
driver, so if we wanted to use the AWS-provided IAM stuff then we'd need to switch to the redshift_connector (which probably would be worthwhile anyways).