[Feature Request]: Handle failed inserts in Storage API sink

apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.

https://beam.apache.org/

Apache License 2.0

7.89k stars 4.27k forks source link

[Feature Request]: Handle failed inserts in Storage API sink #23628

Open reuvenlax opened 2 years ago

reuvenlax commented 2 years ago

What would you like to happen?

The Storage API BigQuery sink already handles rows that do not conform to schema and forwards them to the failed-rows PCollection. We should handle the remaining cases - rows that are too large, and rows that don't conform to any other BigQuery constraints.

Issue Priority

Priority: 2

Issue Component

Component: io-java-gcp

Amar3tto commented 5 months ago

Can this issue be closed or is there something else left?