GoogleCloudDataproc spark-bigquery-connector issues

GoogleCloudDataproc / spark-bigquery-connector

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.

Apache License 2.0

378 stars 198 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Incompatibility with OL 1.24.0+

#1317 tnazarew opened 3 hours ago
0
BQ session expired exception in 6 hours and our spark job gets failed.

#1316 shaikhmohdjaffer closed 1 day ago
1
Fix Markdown heading syntax in PUSHDOWN.md

#1315 nightscape closed 5 days ago
0
"spark.sql.sources.partitionOverwriteMode": "DYNAMIC" - creates additional tables

#1314 MichalBogoryja opened 1 week ago
0
Allowing lower or mixed case partitionOverwriteMode option value

#1313 davidrabinowitz closed 6 days ago
3
Destination table's schema is not compatible with dataframe's schema. Incompatible max length for field: FIELD_NAME, cannot write source field with max length: null to destination field with max length: 8

#1312 kumarshekhar opened 4 weeks ago
1
Improve read session expired error message

#1311 vishalkarve15 closed 3 weeks ago
2
Use date_trunc for DATE field in dynamic partition overwrite

#1310 isha97 closed 1 month ago
3
Spark BQ Connector failing because of Limit - not Quota

#1309 anish97IND closed 1 month ago
1
pyspark issue while connecting the bigquery for read and write in DATAPROC

#1308 san120612 closed 2 weeks ago
5
Respect snapshotTimeMillis when doing optimized count

#1307 tom-s-powell opened 1 month ago
6
GCP Identity federation auth failure with AWS

#1306 sytianhe opened 1 month ago
2
Value at index is null Issue when writing biqquery ga4 data to object storage

#1305 nikosheng closed 1 month ago
2
Spark to Bigquery connectivity through scala code

#1304 Hari0123y closed 1 month ago
2
Support spark.sql.datetime.java8API.enabled

#1303 tom-s-powell closed 1 month ago
6
Bump org.apache.avro:avro from 1.11.3 to 1.11.4 in /spark-bigquery-parent

#1302 dependabot[bot] opened 1 month ago
2
How to enforce BQ connector to NOT use Storage read API in occasion of reaching limits?

#1301 anish97IND closed 1 month ago
2
Internal Error when writing Pyspark df to partitioned Bigquery table (Dataproc)

#1300 lorenh516 closed 1 month ago
6
[Glue 4.0 integration]

#1299 Repomano closed 1 month ago
1
What determines the Query API vs. Storage API to be used?

#1298 whittid4 closed 1 month ago
2
Support for spark version 3.5.0 - support required for UNTYPED_SCALA_UDF version

#1297 VIKCT001 opened 2 months ago
2
Setting a different client api endpoiint for Bigquery

#1296 EmilioIppoliti closed 1 month ago
1
org.apache.spark.sql.AnalysisException: [UNSUPPORTED_DESERIALIZER.DATA_TYPE_MISMATCH] The deserializer is not supported: need a(n) "ARRAY" field but got "MAP<STRING, STRING>".

#1295 VIKCT001 opened 2 months ago
6
exception : BigQueryException: Provided Schema does not match Table

#1294 VIKCT001 closed 2 months ago
4
Issue #1290: Stopped using metadata for optimized count path

#1293 vishalkarve15 closed 1 month ago
11
BigQuery RANGE (and INTERVAL) support

#1292 0rnella opened 2 months ago
1
Error during reading numeric & bignumeric

#1291 flo076 closed 2 months ago
2
Issue with BigQuery counts in Spark after writing to table

#1290 rchynoweth opened 2 months ago
5
Update CHANGES.md

#1289 isha97 closed 2 months ago
0
Change log to debug for exceptions while fetching Identity token

#1288 isha97 closed 2 months ago
0
Upgrade Dependencies

#1287 isha97 closed 2 months ago
2
df.write()not working with Spark-Java code on serverless

#1286 saikrishnab27 closed 2 months ago
1
Unable to write to BigQuery table with require_partition_filter = true when spark.sql.sources.partitionOverwriteMode is set to DYNAMIC

#1285 ktchana closed 1 month ago
2
Making BigQueryClientFactory Kryo serializable

#1284 davidrabinowitz opened 2 months ago
10
Fix readme for configuring bigNumeric defaults

#1283 isha97 closed 2 months ago
0
Issue #1175: Add details to schema mismatch message

#1282 vishalkarve15 closed 2 months ago
6
Configure alternative BigNumeric precision and scale defaults

#1281 isha97 closed 3 months ago
5
Spark structured streaming jobs failing with Bigquery session expired

#1280 caspherola closed 2 months ago
3
Add support for BigQuery tables with Collation

#1279 jster1357 closed 3 months ago
2
Issue with Map Type support (pyspark)

#1278 violetautumn closed 3 months ago
1
`bigquery.tables.create` Permission required on appending data to a BigQuery Table

#1277 smic-datalabs-jdcastro opened 3 months ago
4
Upgrade Dependencies

#1276 isha97 closed 3 months ago
2
Writing spark data with timestamp_ntz/timestamp to bigquery column datetime

#1275 ne250040 closed 3 months ago
1
cloud cost when using spark bq connector

#1274 kaiseu opened 3 months ago
0
Update CHANGES.md for next release

#1273 isha97 closed 3 months ago
0
Move misplaced entry in CHANGES file

#1272 rafalh opened 3 months ago
1
Add to changes.md

#1271 vishalkarve15 closed 3 months ago
2
Issue #1263 : Support ClusteredFields in Direct Write

#1270 isha97 closed 3 months ago
3
Update spark-bigquery-support version

#1269 MattiaIppoliti closed 2 months ago
1
Empty array filtering issue

#1268 vokuk-walkme opened 4 months ago
1