search

spark-redshift-community / spark-redshift

Performant Redshift data source for Apache Spark
Apache License 2.0
135 stars 62 forks source link

DSV2 support for redshift #69

Open karuppayya opened 4 years ago

karuppayya commented 4 years ago

With Spark 3.0(to be released soon) we will have read support for for DSv2. I am planning to use the DSv2 api for Redshift connector. Wanted to check if the community would be interested in this?

smoy commented 4 years ago

yes yes yes. that will be amazing. have been waiting for DSv2 to stabilize the api, which DSv2 doc you are using, i like to take a look as well. Cheers.

karuppayya commented 4 years ago

@smoy I have created an WIP PR. Some code cleanup is pending. It works end to end. Please take a look. https://github.com/spark-redshift-community/spark-redshift/pull/70