search

apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.
https://beam.apache.org/
Apache License 2.0
7.89k stars 4.27k forks source link

Python SDK BigQuery `schemaUpdateOptions` #21141

Open damccorm opened 2 years ago

damccorm commented 2 years ago

I noticed that in the Java SDK schemaUpdateOptions are exposed clearly to the user: https://beam.apache.org/releases/javadoc/2.32.0/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.Write.html

 

Whereas, in the Python SDK this option seems to only be able to be passed through additional_bq_paramaters. 

 

Is this something which should be resolved? Perhaps add the schemaUpdateOptions to https://github.com/apache/beam/blob/b9457b711029051a837255e1bb516e55d32e56d5/sdks/python/apache_beam/io/gcp/bigquery.py#L437 and expose a kwarg to pass them in? 

Imported from Jira BEAM-12994. Original Jira may contain additional context. Reported by: judah.rand.

waltage commented 2 years ago

.take-issue

tsafacjo commented 2 months ago

can I pick it ?