apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.
https://beam.apache.org/
Apache License 2.0
7.88k stars 4.27k forks source link

[Feature Request]: External transforms should be able to ignore optional params of unknown type. #29824

Open robertwb opened 11 months ago

robertwb commented 11 months ago

What would you like to happen?

Ideally SchemaTransforms should limit themselves to language-independent types in their configs, but if they don't it would be good to at least support ignoring the un-interpretable fields if they're optional.

Issue Priority

Priority: 3 (nice-to-have improvement)

Issue Components

tvalentyn commented 9 months ago

cc: @chamikaramj @ahmedabu98