GoogleCloudPlatform / DataflowTemplates

Cloud Dataflow Google-provided templates for solving in-Cloud data tasks
https://cloud.google.com/dataflow/docs/guides/templates/provided-templates
Apache License 2.0
1.12k stars 936 forks source link

Terraform sample for bulk migrations #1629

Closed manitgupta closed 1 month ago

manitgupta commented 2 months ago

This sample presents a scenario of scripting the launch of multiple bulk migration jobs together

Please take a look at a folder structure.

The PR itself contains a dummy terraform.tfvars.

codecov[bot] commented 2 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 45.40%. Comparing base (46f87dd) to head (5d388eb). Report is 26 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1629 +/- ## ============================================ + Coverage 41.49% 45.40% +3.90% + Complexity 2920 713 -2207 ============================================ Files 754 299 -455 Lines 43976 16125 -27851 Branches 4707 1606 -3101 ============================================ - Hits 18247 7321 -10926 + Misses 24190 8264 -15926 + Partials 1539 540 -999 ``` | [Components](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1629/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | Coverage Δ | | |---|---|---| | [spanner-templates](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1629/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `58.82% <ø> (-2.45%)` | :arrow_down: | | [spanner-import-export](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1629/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `∅ <ø> (∅)` | | | [spanner-live-forward-migration](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1629/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `73.56% <ø> (ø)` | | | [spanner-live-reverse-replication](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1629/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `49.04% <ø> (-0.11%)` | :arrow_down: | | [spanner-bulk-migration](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1629/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `81.91% <ø> (ø)` | | [see 483 files with indirect coverage changes](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1629/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform)
manitgupta commented 1 month ago

@bharadwaj-aditya / @VardhanThigle have addressed all comments - PTAL

manitgupta commented 1 month ago

This is already merged as per the merge commit