InfuseAI / piperider

Code review for data in dbt
https://www.piperider.io/
Apache License 2.0
478 stars 23 forks source link

[Feature] Support impersonate service-account for BigQuery #900

Closed qrtt1 closed 10 months ago

qrtt1 commented 10 months ago

PR checklist

What type of PR is this?

feature added

What this PR does / why we need it:

add GCP service-account impersonation support

Which issue(s) this PR fixes:

sc-32423

Special notes for your reviewer:


Configuration example

Piperider will use impersonate_service_account to profile data

jaffle_shop_bq:
  outputs:
    dev:
      dataset: my_dataset
      job_execution_timeout_seconds: 300
      job_retries: 1
      location: US
      method: service-account
      priority: interactive
      project: dev
      threads: 1
      type: bigquery
      keyfile: service-account.json
      impersonate_service_account: someone_has_the_permission@dev.iam.gserviceaccount.com
  target: dev
codecov[bot] commented 10 months ago

Codecov Report

Merging #900 (a713e14) into main (a1a9823) will decrease coverage by 0.18%. The diff coverage is 4.76%.

@@            Coverage Diff             @@
##             main     #900      +/-   ##
==========================================
- Coverage   48.20%   48.02%   -0.18%     
==========================================
  Files          76       77       +1     
  Lines       10645    10685      +40     
==========================================
+ Hits         5131     5132       +1     
- Misses       5514     5553      +39     
Files Coverage Δ
piperider_cli/datasource/bigquery.py 19.88% <25.00%> (-0.69%) :arrow_down:
piperider_cli/datasource/bigquery_patch.py 0.00% <0.00%> (ø)

... and 1 file with indirect coverage changes

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

popcornylu commented 10 months ago

Fix #899