GoogleCloudPlatform / DataflowTemplates

Cloud Dataflow Google-provided templates for solving in-Cloud data tasks
https://cloud.google.com/dataflow/docs/guides/templates/provided-templates
Apache License 2.0
1.12k stars 936 forks source link

Refactor Avro code for Kafka templates #1648

Closed AnandInguva closed 1 month ago

codecov[bot] commented 1 month ago

Codecov Report

Attention: Patch coverage is 0% with 283 lines in your changes missing coverage. Please review.

Project coverage is 41.27%. Comparing base (c06ab9a) to head (65d2c5c). Report is 6 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1648 +/- ## ============================================ - Coverage 41.27% 41.27% -0.01% + Complexity 2928 2925 -3 ============================================ Files 760 762 +2 Lines 44333 44341 +8 Branches 4745 4749 +4 ============================================ + Hits 18299 18301 +2 - Misses 24491 24494 +3 - Partials 1543 1546 +3 ``` | [Components](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1648/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | Coverage Δ | | |---|---|---| | [spanner-templates](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1648/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `61.34% <ø> (-0.02%)` | :arrow_down: | | [spanner-import-export](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1648/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `64.36% <ø> (-0.09%)` | :arrow_down: | | [spanner-live-forward-migration](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1648/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `73.88% <ø> (ø)` | | | [spanner-live-reverse-replication](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1648/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `49.66% <ø> (ø)` | | | [spanner-bulk-migration](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1648/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `82.07% <ø> (+0.03%)` | :arrow_up: | | [Files](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1648?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | Coverage Δ | | |---|---|---| | [...le/cloud/teleport/v2/templates/KafkaToGcsFlex.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1648?src=pr&el=tree&filepath=v2%2Fkafka-to-gcs%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Ftemplates%2FKafkaToGcsFlex.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIva2Fma2EtdG8tZ2NzL3NyYy9tYWluL2phdmEvY29tL2dvb2dsZS9jbG91ZC90ZWxlcG9ydC92Mi90ZW1wbGF0ZXMvS2Fma2FUb0djc0ZsZXguamF2YQ==) | `0.00% <ø> (ø)` | | | [...e/cloud/teleport/v2/transforms/WriteTransform.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1648?src=pr&el=tree&filepath=v2%2Fkafka-to-gcs%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Ftransforms%2FWriteTransform.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIva2Fma2EtdG8tZ2NzL3NyYy9tYWluL2phdmEvY29tL2dvb2dsZS9jbG91ZC90ZWxlcG9ydC92Mi90cmFuc2Zvcm1zL1dyaXRlVHJhbnNmb3JtLmphdmE=) | `0.00% <0.00%> (ø)` | | | [...port/v2/kafka/transforms/AvroDynamicTransform.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1648?src=pr&el=tree&filepath=v2%2Fkafka-common%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Fkafka%2Ftransforms%2FAvroDynamicTransform.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIva2Fma2EtY29tbW9uL3NyYy9tYWluL2phdmEvY29tL2dvb2dsZS9jbG91ZC90ZWxlcG9ydC92Mi9rYWZrYS90cmFuc2Zvcm1zL0F2cm9EeW5hbWljVHJhbnNmb3JtLmphdmE=) | `0.00% <0.00%> (ø)` | | | [...ud/teleport/v2/kafka/transforms/AvroTransform.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1648?src=pr&el=tree&filepath=v2%2Fkafka-common%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Fkafka%2Ftransforms%2FAvroTransform.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIva2Fma2EtY29tbW9uL3NyYy9tYWluL2phdmEvY29tL2dvb2dsZS9jbG91ZC90ZWxlcG9ydC92Mi9rYWZrYS90cmFuc2Zvcm1zL0F2cm9UcmFuc2Zvcm0uamF2YQ==) | `0.00% <0.00%> (ø)` | | | [...oud/teleport/v2/transforms/AvroWriteTransform.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1648?src=pr&el=tree&filepath=v2%2Fkafka-to-gcs%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Ftransforms%2FAvroWriteTransform.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIva2Fma2EtdG8tZ2NzL3NyYy9tYWluL2phdmEvY29tL2dvb2dsZS9jbG91ZC90ZWxlcG9ydC92Mi90cmFuc2Zvcm1zL0F2cm9Xcml0ZVRyYW5zZm9ybS5qYXZh) | `0.00% <0.00%> (ø)` | | | [...s/KafkaRecordToGenericRecordFailsafeElementFn.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1648?src=pr&el=tree&filepath=v2%2Fkafka-common%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Fkafka%2Ftransforms%2FKafkaRecordToGenericRecordFailsafeElementFn.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIva2Fma2EtY29tbW9uL3NyYy9tYWluL2phdmEvY29tL2dvb2dsZS9jbG91ZC90ZWxlcG9ydC92Mi9rYWZrYS90cmFuc2Zvcm1zL0thZmthUmVjb3JkVG9HZW5lcmljUmVjb3JkRmFpbHNhZmVFbGVtZW50Rm4uamF2YQ==) | `0.00% <0.00%> (ø)` | | | [...oud/teleport/v2/templates/KafkaToBigQueryFlex.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1648?src=pr&el=tree&filepath=v2%2Fkafka-to-bigquery%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Ftemplates%2FKafkaToBigQueryFlex.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIva2Fma2EtdG8tYmlncXVlcnkvc3JjL21haW4vamF2YS9jb20vZ29vZ2xlL2Nsb3VkL3RlbGVwb3J0L3YyL3RlbXBsYXRlcy9LYWZrYVRvQmlnUXVlcnlGbGV4LmphdmE=) | `0.00% <0.00%> (ø)` | | | [...oud/teleport/v2/transforms/BigQueryWriteUtils.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1648?src=pr&el=tree&filepath=v2%2Fkafka-to-bigquery%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fv2%2Ftransforms%2FBigQueryWriteUtils.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djIva2Fma2EtdG8tYmlncXVlcnkvc3JjL21haW4vamF2YS9jb20vZ29vZ2xlL2Nsb3VkL3RlbGVwb3J0L3YyL3RyYW5zZm9ybXMvQmlnUXVlcnlXcml0ZVV0aWxzLmphdmE=) | `0.00% <0.00%> (ø)` | | ... and [4 files with indirect coverage changes](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1648/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform)