spotify / spark-bigquery

Google BigQuery support for Spark, SQL, and DataFrames
Apache License 2.0
155 stars 52 forks source link

Bug fixes on 0.2.2 + adding standard sql support + documentation #71

Open lverzosa opened 5 years ago

lverzosa commented 5 years ago

Hi everybody,

LMK if you need more info, but effectively I just made changes to 0.2.2. There were a lot of library conflicts with the latest versions of the libraries since then so I've reverted those changes back.

In this PR:

Revert libraries to 0.2.2 version and scala back down to 2.11 (as 2.12 won't be supported until Spark 3.0) Fixed GoogleCredential for bigQuerySelect to use credentials file (in Json) provided by setGcpJsonKeyFile() instead of GoogleCredential.getApplicationDefault() Added flag for standard sql (while not breaking API for legacy SQL) Documentation changes (removed reference to Dataproc since I believe google provides an up to date connector that works on Dataproc) Lorenz