search

transferwise / pipelinewise-target-redshift

Singer.io Target for Amazon Redshift - PipelineWise compatible
https://transferwise.github.io/pipelinewise/
Other
12 stars 65 forks source link

Redshift IAM Authentication #233

Open hazmei opened 1 year ago

hazmei commented 1 year ago

Hi,

It seems that the target-redshift plugin for meltano does not have IAM authentication for connecting to the redshift cluster.

This would be great as we will be able to reuse the IAM role that the meltano service is already using for connecting to the AWS S3 bucket. It also helps that with IAM authentication to redshift, we will not need to have the redshift DB user password stored anywhere in the meltano service.

Is there any future plans to introduce the feature so we're able to connect to redshift via IAM authentication?

jpuris commented 1 year ago

Actually nvm.. the PR #231 does not fix this. It is a slightly different problem. Said that, it should not be difficult to implement at all..

~You are looking to use the IAM role arn in CREDENTIALS opt in COPY statement the target does, right?~

edit: I see what you mean.. you want to use the same IAM Role it would use for the COPY from S3 via aws_redshift_copy_role_arn.

The challenge here will be to have it merged as it seems this repository has not seen any updates to master since some years now :(

jpuris commented 1 year ago

@hazmei Could you please elaborate on your exact use case 🤔

Are you looking for functionality that will retrieve the temporary credentials for your IAM user / Role and then use these credentials to log into RS?

AWS docs in question (Redshift service) https://docs.aws.amazon.com/redshift/latest/mgmt/generating-iam-credentials-cli-api.html

Also I suppose we want to support both Redshift and Redshift Serverless use cases?