Google authentication difference between table and SQL queries

miraisolutions / sparkbq

Sparklyr extension package to connect to Google BigQuery

GNU General Public License v3.0

19 stars 3 forks source link

Google authentication difference between table and SQL queries #1

Closed martinstuder closed 6 years ago

martinstuder commented 7 years ago

When using spark_read_bigquery there seems to be a difference between querying a table vs querying using a SQL query. Specifically, table queries require setting config[["spark.hadoop.google.cloud.auth.service.account.json.keyfile"]] = gcpJsonKeyfile while SQL queries seem to require setting the GOOGLE_APPLICATION_CREDENTIALS environment variable.

martinstuder commented 6 years ago

No longer an issue with the latest version of sparkbq which depends on https://github.com/miraisolutions/spark-bigquery. See notes on authentication in README.md