I noticed this when testing with Managed BigQueryIO in Python SDK:
options = PipelineOptions([
"--project=google.com:clouddfe",
"--region=us-central1",
"--temp_location=gs://ahmedabualsaud-wordcount/tmp",
])
with beam.Pipeline(options=options) as p:
(p
| beam.Create([beam.Row(int=999)])
| beam.managed.Write("bigquery",
config={"table": "google.com:clouddfe.ahmedabualsaud.test_table"},
expansion_service="localhost:12345"
))
I would expect these pipeline options to be picked up on Java's side. With some logging, I see the Java SDK recognizing the options at expand time in PTransforms (i.e. expand()), but not at runtime in DoFns (i.e. @ProcessElement).
Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
What happened?
I noticed this when testing with Managed BigQueryIO in Python SDK:
I would expect these pipeline options to be picked up on Java's side. With some logging, I see the Java SDK recognizing the options at expand time in PTransforms (i.e. expand()), but not at runtime in DoFns (i.e. @ProcessElement).
Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
Issue Components