search

spotify / spark-bigquery

Google BigQuery support for Spark, SQL, and DataFrames
Apache License 2.0
155 stars 52 forks source link

Exporting an empty table from BigQuery throws an `empty collection` exception #24

Open yuehuaz opened 7 years ago

yuehuaz commented 7 years ago

Seems like it always tries to get the schema from the first record: https://github.com/spotify/spark-bigquery/blob/master/src/main/scala/com/spotify/spark/bigquery/package.scala#L122

nevillelyh commented 7 years ago

Thanks for reporting. Since we're using GenericRecord, there's no way to get schema from the type and we have to read it from Avro file header instead.

I'm out till Dec and may not look at this at time soon. You wanna give it a shot and submit a PR?

yuehuaz commented 7 years ago

Gotcha. I am using a workaround by returning an option instead.