spotify / spark-bigquery

Google BigQuery support for Spark, SQL, and DataFrames
Apache License 2.0
155 stars 52 forks source link

Query DATE_RANGE or table wildcards #42

Open juliankeppel opened 7 years ago

juliankeppel commented 7 years ago

I want to use the BigQuery connector to load data from multiple tables at a time. I tried to use DATERANGE functions and table wildcards, but both doesn't work and I get the following error: `Invalid datasetAndTableString '.'; must match regex '[a-zA-Z0-9]+.[a-zA-Z0-9_]+'.`

I want to load the raw data, so I can't use sqlContext.bigQuerySelect() and provide SQL statement, because then I would have to unnest the nested data.

tl;dr: Is it possible to load raw data from multiple tables at one time with something like DATE_RANGE or wildcard functions which are provided by BigQuery itself?