Suggested replacements to this project?

spotify / spark-bigquery

Google BigQuery support for Spark, SQL, and DataFrames

Apache License 2.0

155 stars 52 forks source link

Suggested replacements to this project? #66

Closed alexklibisz closed 5 years ago

alexklibisz commented 5 years ago

The README suggests that this project is no longer used widely within Spotify. Are there suggested replacements for transferring data from Bigquery into spark for big processing jobs? Or is this workflow in general outdated? So far this is the best piece of OSS I've found for the purpose.

pkallos commented 5 years ago

@GoogleCloudPlatform recently released GoogleCloudPlatform/spark-bigquery-connector with read support, assume write support is in the works with GoogleCloudPlatform/spark-bigquery-connector#7

alexklibisz commented 5 years ago

Thanks for following up. That is pretty recent and from scanning the readme seems to do what I need.