Open fbbergamo opened 6 years ago
Sorry for the late reply @fbbergamo, It is a good idea, there isnt a specific reason we cant do this, as far as I know its only the redshift connector that uses this, so there isnt really a spark standard to writing a connector
I think we did start writing a class but it really didn't go anywhere
You can look here and send a pr
@samelamin thanks, I will work on and let you know. :)
Is there any problem to implement the writing to Bigquery using DataFrameWriter,
What Do you guys think to implement like the redshift lib?
https://github.com/databricks/spark-redshift/blob/master/src/main/scala/com/databricks/spark/redshift/DefaultSource.scala#L68
I want to use the spark like other formats:
This will help to use this library using the same interface that is already build in my project.
Is there any other way to achieve this?