Support impersonation for BigQuery clients for source and target queries

[dmedora]

Today, if the source or target system is BigQuery, the BigQuery client is run using application default credentials or a provided service account keyfile.

Feature request: Provide a way to pass an identity to use for service account impersonation.

[nick-redfearn]

Has there been any recent consideration for this feature? It would be a great addition

[nehanene15]

@nick-redfearn Yeah - we recently discussed it in #768 - we do support passing impersonated credentials via Composer/Airflow/Python invocations. Would you be interested in supporting impersonated credentials via the CLI as well when creating a BQ connection?

[nick-redfearn]

@nick-redfearn Yeah - we recently discussed it in #768 - we do support passing impersonated credentials via Composer/Airflow/Python invocations. Would you be interested in supporting impersonated credentials via the CLI as well when creating a BQ connection?

@nehanene15 - Yes please, Im looking at implementing via a Docker image running in Kubernetes pods

[nehanene15]

@nick-redfearn By default, DVT uses the BQ client library which uses application default credentials. You should be able to impersonate credentials when setting up an ADC file as described here. Is this something that would be feasible for a GKE pod approach?

Because users can't pass in Credentials objects into a CLI command i.e. data-validation connections add -c bq BigQuery --project-id my-project ... creating impersonated credentials via the CLI becomes difficult.

[nick-redfearn]

@nehanene15 Understood. Thanks for the info. Will give this approach a try with my implementation.