search

spark-redshift-community / spark-redshift

Performant Redshift data source for Apache Spark
Apache License 2.0
136 stars 63 forks source link

Unable to use IAM credentials #62

Open Gauravshah opened 4 years ago

Gauravshah commented 4 years ago

Clone of issue : https://github.com/databricks/spark-redshift/issues/370

Newer versions of the jdbc driver allow for use of IAM credentials instead of username/password. [1]

This is preferable to storing and passing around credentials.

It looks like the parameter validation in spark-redshift should be updated to reflect the new authentication method. [2]

[1] http://docs.aws.amazon.com/redshift/latest/mgmt/generating-iam-credentials-configure-jdbc-odbc.html [2] https://github.com/databricks/spark-redshift/blob/master/src/main/scala/com/databricks/spark/redshift/Parameters.scala#L71

lucagiovagnoli commented 4 years ago

Hi @Gauravshah, we removed tests and support for IAM credentials in 4.0.0, check out the CHANGELOG here

  • STS and IAM authentication support has been dropped.

It was a lot of work to get to 4.0.0 and we had to drop something. If you'd like to fix IAM support, I'll be happy to review your PR