GoogleCloudDataproc / spark-bigquery-connector

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.
Apache License 2.0
378 stars 198 forks source link

Spark to Bigquery connectivity through scala code #1304

Closed Hari0123y closed 1 month ago

Hari0123y commented 1 month ago

Df = spark.read.format(“bigquery”).option(“parentProject”, project id).option(“viewsEnabled”, “true”).option(“materializationDataset”, datasetid).option(“query”, sqlquery).load()

while running this one through scala code getting this following error

Error creating destination table using the following query: [sqlquery]

davidrabinowitz commented 1 month ago

Please share the full stack trace and error

Hari0123y commented 1 month ago

Issue is resolved now @davidrabinowitz , Service account doesn’t have bigquery job user access in project in the IAM . So it was failing with above error