search

samelamin / spark-bigquery

Google BigQuery support for Spark, Structured Streaming, SQL, and DataFrames with easy Databricks integration.
Apache License 2.0
70 stars 28 forks source link

Fix #56: Exception when biQuerySelect Returns Zero Rows #82

Closed sujeet100 closed 4 years ago

sujeet100 commented 4 years ago

This pull requests fixed the bug #56 When bigquery returns empty result then the bigQuerySelect throws an exception while trying to figure out the schema.

Check if data is empty before converting dataframe using schema else return empty dataframe. the isEmpty() check cannot be done directly on the RDD as it fails with serialization issue. Hence the isEmpty() check is applied after map operation to schema.

samelamin commented 4 years ago

Thanks for the PR! 👍

sagarkbhatt commented 4 years ago

Is this released in latest version?

samelamin commented 4 years ago

It should be yes

sagarkbhatt commented 4 years ago

Seems like it is not released in latest version. Could you please tell which version to use?

sujeet100 commented 4 years ago

@samelamin Seems like 0.2.6 is the latest one available in maven repo. https://repo1.maven.org/maven2/com/github/samelamin/spark-bigquery_2.11/ Could you please publish a new version of the library?

samelamin commented 4 years ago

No Worries i will do that this weekend hopefully