GoogleCloudPlatform / DataflowTemplates

Cloud Dataflow Google-provided templates for solving in-Cloud data tasks
https://cloud.google.com/dataflow/docs/guides/templates/provided-templates
Apache License 2.0
1.16k stars 976 forks source link

refactor YAML_Template to yaml module #1984

Open Polber opened 3 weeks ago

Polber commented 3 weeks ago

This PR refactors the organization of the Yaml_Template and its related files by creating a new yaml module.

This PR also adds a new workflow for triggering YAML-related tests separately. This will be useful as new ITs are added to keep from slowing down the workflow for Java PRs

codecov[bot] commented 3 weeks ago

Codecov Report

Attention: Patch coverage is 0% with 7 lines in your changes missing coverage. Please review.

Project coverage is 45.26%. Comparing base (ba67089) to head (6ca0025). Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
.../java/org/apache/beam/it/gcp/TemplateTestBase.java 0.00% 7 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1984 +/- ## ============================================ - Coverage 45.27% 45.26% -0.02% - Complexity 3632 3956 +324 ============================================ Files 836 836 Lines 49872 49877 +5 Branches 5253 5255 +2 ============================================ - Hits 22579 22576 -3 - Misses 25618 25627 +9 + Partials 1675 1674 -1 ``` | [Components](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1984/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | Coverage Δ | | |---|---|---| | [spanner-templates](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1984/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `66.62% <ø> (ø)` | | | [spanner-import-export](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1984/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `64.18% <ø> (ø)` | | | [spanner-live-forward-migration](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1984/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `75.86% <ø> (ø)` | | | [spanner-live-reverse-replication](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1984/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `76.63% <ø> (ø)` | | | [spanner-bulk-migration](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1984/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `86.42% <ø> (ø)` | | | [Files with missing lines](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1984?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | Coverage Δ | | |---|---|---| | [.../java/org/apache/beam/it/gcp/TemplateTestBase.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1984?src=pr&el=tree&filepath=it%2Fgoogle-cloud-platform%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fbeam%2Fit%2Fgcp%2FTemplateTestBase.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-aXQvZ29vZ2xlLWNsb3VkLXBsYXRmb3JtL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9iZWFtL2l0L2djcC9UZW1wbGF0ZVRlc3RCYXNlLmphdmE=) | `0.00% <0.00%> (ø)` | | ... and [3 files with indirect coverage changes](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1984/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform)