search

GoogleCloudPlatform / DataflowTemplates

Cloud Dataflow Google-provided templates for solving in-Cloud data tasks
https://cloud.google.com/dataflow/docs/guides/templates/provided-templates
Apache License 2.0
1.16k stars 976 forks source link

Refactor source specific code - reverse replication #2000

Closed shreyakhajanchi closed 2 days ago

codecov[bot] commented 2 weeks ago

Codecov Report

Attention: Patch coverage is 78.31325% with 36 lines in your changes missing coverage. Please review.

Project coverage is 45.57%. Comparing base (cb9b943) to head (b747d4f). Report is 17 commits behind head on main.

Files with missing lines Patch % Lines
...lates/dbutils/connection/JdbcConnectionHelper.java 52.00% 12 Missing :warning:
...cloud/teleport/v2/templates/SpannerToSourceDb.java 0.00% 8 Missing :warning:
...ates/dbutils/processor/SourceProcessorFactory.java 89.58% 4 Missing and 1 partial :warning:
...2/templates/dbutils/processor/SourceProcessor.java 84.21% 3 Missing :warning:
...leport/v2/templates/transforms/SourceWriterFn.java 66.66% 3 Missing :warning:
...rt/v2/templates/dbutils/dml/MySQLDMLGenerator.java 95.65% 2 Missing :warning:
...v2/templates/transforms/SourceWriterTransform.java 0.00% 2 Missing :warning:
...plates/dbutils/processor/InputRecordProcessor.java 83.33% 0 Missing and 1 partial :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2000 +/- ## ============================================ + Coverage 45.30% 45.57% +0.26% - Complexity 3633 3721 +88 ============================================ Files 836 847 +11 Lines 49922 50166 +244 Branches 5260 5272 +12 ============================================ + Hits 22619 22863 +244 - Misses 25627 25628 +1 + Partials 1676 1675 -1 ``` | [Components](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | Coverage Δ | | |---|---|---| | [spanner-templates](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `67.00% <78.31%> (+0.33%)` | :arrow_up: | | [spanner-import-export](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `64.14% <ø> (-0.07%)` | :arrow_down: | | [spanner-live-forward-migration](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `75.91% <ø> (+0.02%)` | :arrow_up: | | [spanner-live-reverse-replication](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `76.87% <78.31%> (+0.22%)` | :arrow_up: | | [spanner-bulk-migration](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `86.56% <ø> (+0.21%)` | :arrow_up: | | [Files with missing lines](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | Coverage Δ | | |---|---|---| | [...eport/v2/templates/dbutils/dao/source/JdbcDao.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000?src=pr&el=tree&filepath=v2%2Fspanner-to-sourcedb%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Ftemplates%2Fdbutils%2Fdao%2Fsource%2FJdbcDao.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIvc3Bhbm5lci10by1zb3VyY2VkYi9zcmMvbWFpbi9qYXZhL2NvbS9nb29nbGUvY2xvdWQvdGVsZXBvcnQvdjIvdGVtcGxhdGVzL2RidXRpbHMvZGFvL3NvdXJjZS9KZGJjRGFvLmphdmE=) | `100.00% <100.00%> (ø)` | | | [...t/v2/templates/dbutils/dao/spanner/SpannerDao.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000?src=pr&el=tree&filepath=v2%2Fspanner-to-sourcedb%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Ftemplates%2Fdbutils%2Fdao%2Fspanner%2FSpannerDao.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIvc3Bhbm5lci10by1zb3VyY2VkYi9zcmMvbWFpbi9qYXZhL2NvbS9nb29nbGUvY2xvdWQvdGVsZXBvcnQvdjIvdGVtcGxhdGVzL2RidXRpbHMvZGFvL3NwYW5uZXIvU3Bhbm5lckRhby5qYXZh) | `63.63% <ø> (ø)` | | | [...plates/dbutils/processor/InputRecordProcessor.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000?src=pr&el=tree&filepath=v2%2Fspanner-to-sourcedb%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Ftemplates%2Fdbutils%2Fprocessor%2FInputRecordProcessor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIvc3Bhbm5lci10by1zb3VyY2VkYi9zcmMvbWFpbi9qYXZhL2NvbS9nb29nbGUvY2xvdWQvdGVsZXBvcnQvdjIvdGVtcGxhdGVzL2RidXRpbHMvcHJvY2Vzc29yL0lucHV0UmVjb3JkUHJvY2Vzc29yLmphdmE=) | `90.32% <83.33%> (ø)` | | | [...rt/v2/templates/dbutils/dml/MySQLDMLGenerator.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000?src=pr&el=tree&filepath=v2%2Fspanner-to-sourcedb%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Ftemplates%2Fdbutils%2Fdml%2FMySQLDMLGenerator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIvc3Bhbm5lci10by1zb3VyY2VkYi9zcmMvbWFpbi9qYXZhL2NvbS9nb29nbGUvY2xvdWQvdGVsZXBvcnQvdjIvdGVtcGxhdGVzL2RidXRpbHMvZG1sL015U1FMRE1MR2VuZXJhdG9yLmphdmE=) | `97.93% <95.65%> (ø)` | | | [...v2/templates/transforms/SourceWriterTransform.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000?src=pr&el=tree&filepath=v2%2Fspanner-to-sourcedb%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Ftemplates%2Ftransforms%2FSourceWriterTransform.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIvc3Bhbm5lci10by1zb3VyY2VkYi9zcmMvbWFpbi9qYXZhL2NvbS9nb29nbGUvY2xvdWQvdGVsZXBvcnQvdjIvdGVtcGxhdGVzL3RyYW5zZm9ybXMvU291cmNlV3JpdGVyVHJhbnNmb3JtLmphdmE=) | `0.00% <0.00%> (ø)` | | | [...2/templates/dbutils/processor/SourceProcessor.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000?src=pr&el=tree&filepath=v2%2Fspanner-to-sourcedb%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Ftemplates%2Fdbutils%2Fprocessor%2FSourceProcessor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIvc3Bhbm5lci10by1zb3VyY2VkYi9zcmMvbWFpbi9qYXZhL2NvbS9nb29nbGUvY2xvdWQvdGVsZXBvcnQvdjIvdGVtcGxhdGVzL2RidXRpbHMvcHJvY2Vzc29yL1NvdXJjZVByb2Nlc3Nvci5qYXZh) | `84.21% <84.21%> (ø)` | | | [...leport/v2/templates/transforms/SourceWriterFn.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000?src=pr&el=tree&filepath=v2%2Fspanner-to-sourcedb%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Ftemplates%2Ftransforms%2FSourceWriterFn.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIvc3Bhbm5lci10by1zb3VyY2VkYi9zcmMvbWFpbi9qYXZhL2NvbS9nb29nbGUvY2xvdWQvdGVsZXBvcnQvdjIvdGVtcGxhdGVzL3RyYW5zZm9ybXMvU291cmNlV3JpdGVyRm4uamF2YQ==) | `85.29% <66.66%> (+5.66%)` | :arrow_up: | | [...ates/dbutils/processor/SourceProcessorFactory.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000?src=pr&el=tree&filepath=v2%2Fspanner-to-sourcedb%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Ftemplates%2Fdbutils%2Fprocessor%2FSourceProcessorFactory.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIvc3Bhbm5lci10by1zb3VyY2VkYi9zcmMvbWFpbi9qYXZhL2NvbS9nb29nbGUvY2xvdWQvdGVsZXBvcnQvdjIvdGVtcGxhdGVzL2RidXRpbHMvcHJvY2Vzc29yL1NvdXJjZVByb2Nlc3NvckZhY3RvcnkuamF2YQ==) | `89.58% <89.58%> (ø)` | | | [...cloud/teleport/v2/templates/SpannerToSourceDb.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000?src=pr&el=tree&filepath=v2%2Fspanner-to-sourcedb%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Ftemplates%2FSpannerToSourceDb.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIvc3Bhbm5lci10by1zb3VyY2VkYi9zcmMvbWFpbi9qYXZhL2NvbS9nb29nbGUvY2xvdWQvdGVsZXBvcnQvdjIvdGVtcGxhdGVzL1NwYW5uZXJUb1NvdXJjZURiLmphdmE=) | `0.00% <0.00%> (ø)` | | | [...lates/dbutils/connection/JdbcConnectionHelper.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000?src=pr&el=tree&filepath=v2%2Fspanner-to-sourcedb%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Ftemplates%2Fdbutils%2Fconnection%2FJdbcConnectionHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIvc3Bhbm5lci10by1zb3VyY2VkYi9zcmMvbWFpbi9qYXZhL2NvbS9nb29nbGUvY2xvdWQvdGVsZXBvcnQvdjIvdGVtcGxhdGVzL2RidXRpbHMvY29ubmVjdGlvbi9KZGJjQ29ubmVjdGlvbkhlbHBlci5qYXZh) | `32.65% <52.00%> (ø)` | | ... and [32 files with indirect coverage changes](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/2000/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform)