[Feature Request]: Enable CDC configuration for cross language StorageWrite API usage - Githubissues

apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.

https://beam.apache.org/

Apache License 2.0

7.88k stars 4.26k forks source link

[Feature Request]: Enable CDC configuration for cross language StorageWrite API usage #32527

Closed prodriguezdefino closed 1 month ago

prodriguezdefino commented 1 month ago

What would you like to happen?

As an Apache Beam Python SDK developer, when wanting to write to BigQuery using its CDC capabilities, I would like to be able to configure:

what is the row mutation information, so BigQuery can decide how to modify the data on the table
and in case I want the table to be created, what is the primary key of the table data will be written to.

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

[X] Component: Python SDK
[X] Component: Java SDK
[ ] Component: Go SDK
[ ] Component: Typescript SDK
[X] Component: IO connector
[ ] Component: Beam YAML
[ ] Component: Beam examples
[ ] Component: Beam playground
[ ] Component: Beam katas
[ ] Component: Website
[ ] Component: Infrastructure
[ ] Component: Spark Runner
[ ] Component: Flink Runner
[ ] Component: Samza Runner
[ ] Component: Twister2 Runner
[ ] Component: Hazelcast Jet Runner
[ ] Component: Google Cloud Dataflow Runner

prodriguezdefino commented 1 month ago

.take-issue