Open kisssam opened 1 year ago
It seems that we need a similar functionality to support ReadFromBigQuery transform for Python examples in Playground.
Our examples need to read the sample data from a different GCP project (clouddataflow-readonly), but use our GCP project for running the job, as we cannot give any permissions for our service account to create jobs in the clouddataflow-readonly one.
This already works in Java (https://play-dev.beam.apache.org/?path=SDK_JAVA_MinimalBigQueryTornadoes&sdk=java).
What would you like to happen?
The ReadFromBigQuery method should provide an option to set the query execution project:
load_job_project_id
parameter in WriteToBigQuery in the Python SDK.bigQueryProject
parameter in the Java SDK .Currently, the project is inferred from the project running the pipeline or the project which contains the temporary dataset, if the temp dataset is set.
Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
Issue Components