search

GoogleCloudPlatform / DataflowTemplates

Cloud Dataflow Google-provided templates for solving in-Cloud data tasks
https://cloud.google.com/dataflow/docs/guides/templates/provided-templates
Apache License 2.0
1.11k stars 931 forks source link

Formatting changes to templates #1667

Open sharan-malyala opened 3 weeks ago

sharan-malyala commented 3 weeks ago

Fixed formatting.

sharan-malyala commented 1 week ago

@rszper Please confirm if these changes are good ? If yes, I will proceed to generate readme docs.

codecov[bot] commented 1 hour ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 42.26%. Comparing base (ca15c0e) to head (2d90f45). Report is 92 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1667 +/- ## ============================================ + Coverage 41.40% 42.26% +0.85% - Complexity 2937 3471 +534 ============================================ Files 770 791 +21 Lines 44937 46137 +1200 Branches 4805 4935 +130 ============================================ + Hits 18608 19501 +893 - Misses 24761 25045 +284 - Partials 1568 1591 +23 ``` | [Components](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | Coverage Δ | | |---|---|---| | [spanner-templates](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `63.63% <ø> (+2.40%)` | :arrow_up: | | [spanner-import-export](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `64.42% <ø> (+0.02%)` | :arrow_up: | | [spanner-live-forward-migration](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `74.97% <ø> (+0.93%)` | :arrow_up: | | [spanner-live-reverse-replication](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `51.44% <ø> (+1.02%)` | :arrow_up: | | [spanner-bulk-migration](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `83.14% <ø> (+2.57%)` | :arrow_up: | | [Files](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | Coverage Δ | | |---|---|---| | [...google/cloud/teleport/bigtable/AvroToBigtable.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667?src=pr&el=tree&filepath=v1%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fbigtable%2FAvroToBigtable.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djEvc3JjL21haW4vamF2YS9jb20vZ29vZ2xlL2Nsb3VkL3RlbGVwb3J0L2JpZ3RhYmxlL0F2cm9Ub0JpZ3RhYmxlLmphdmE=) | `62.26% <ø> (ø)` | | | [...google/cloud/teleport/bigtable/BigtableToJson.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667?src=pr&el=tree&filepath=v1%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fbigtable%2FBigtableToJson.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djEvc3JjL21haW4vamF2YS9jb20vZ29vZ2xlL2Nsb3VkL3RlbGVwb3J0L2JpZ3RhYmxlL0JpZ3RhYmxlVG9Kc29uLmphdmE=) | `63.33% <ø> (ø)` | | | [...gle/cloud/teleport/bigtable/BigtableToParquet.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667?src=pr&el=tree&filepath=v1%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fbigtable%2FBigtableToParquet.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djEvc3JjL21haW4vamF2YS9jb20vZ29vZ2xlL2Nsb3VkL3RlbGVwb3J0L2JpZ3RhYmxlL0JpZ3RhYmxlVG9QYXJxdWV0LmphdmE=) | `41.86% <ø> (ø)` | | | [.../teleport/bigtable/BigtableToVectorEmbeddings.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667?src=pr&el=tree&filepath=v1%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fbigtable%2FBigtableToVectorEmbeddings.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djEvc3JjL21haW4vamF2YS9jb20vZ29vZ2xlL2Nsb3VkL3RlbGVwb3J0L2JpZ3RhYmxlL0JpZ3RhYmxlVG9WZWN0b3JFbWJlZGRpbmdzLmphdmE=) | `78.33% <ø> (ø)` | | | [...e/cloud/teleport/bigtable/CassandraToBigtable.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667?src=pr&el=tree&filepath=v1%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fbigtable%2FCassandraToBigtable.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djEvc3JjL21haW4vamF2YS9jb20vZ29vZ2xlL2Nsb3VkL3RlbGVwb3J0L2JpZ3RhYmxlL0Nhc3NhbmRyYVRvQmlndGFibGUuamF2YQ==) | `0.00% <ø> (ø)` | | | [...gle/cloud/teleport/bigtable/ParquetToBigtable.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667?src=pr&el=tree&filepath=v1%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fbigtable%2FParquetToBigtable.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djEvc3JjL21haW4vamF2YS9jb20vZ29vZ2xlL2Nsb3VkL3RlbGVwb3J0L2JpZ3RhYmxlL1BhcnF1ZXRUb0JpZ3RhYmxlLmphdmE=) | `57.14% <ø> (ø)` | | | [.../google/cloud/teleport/spanner/ExportPipeline.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667?src=pr&el=tree&filepath=v1%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fspanner%2FExportPipeline.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djEvc3JjL21haW4vamF2YS9jb20vZ29vZ2xlL2Nsb3VkL3RlbGVwb3J0L3NwYW5uZXIvRXhwb3J0UGlwZWxpbmUuamF2YQ==) | `0.00% <ø> (ø)` | | | [...gle/cloud/teleport/spanner/TextImportPipeline.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667?src=pr&el=tree&filepath=v1%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Fspanner%2FTextImportPipeline.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djEvc3JjL21haW4vamF2YS9jb20vZ29vZ2xlL2Nsb3VkL3RlbGVwb3J0L3NwYW5uZXIvVGV4dEltcG9ydFBpcGVsaW5lLmphdmE=) | `0.00% <ø> (ø)` | | | [...e/cloud/teleport/templates/BigQueryToTFRecord.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667?src=pr&el=tree&filepath=v1%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Ftemplates%2FBigQueryToTFRecord.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djEvc3JjL21haW4vamF2YS9jb20vZ29vZ2xlL2Nsb3VkL3RlbGVwb3J0L3RlbXBsYXRlcy9CaWdRdWVyeVRvVEZSZWNvcmQuamF2YQ==) | `19.69% <ø> (ø)` | | | [...teleport/templates/DLPTextToBigQueryStreaming.java](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667?src=pr&el=tree&filepath=v1%2Fsrc%2Fmain%2Fjava%2Fcom%2Fgoogle%2Fcloud%2Fteleport%2Ftemplates%2FDLPTextToBigQueryStreaming.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform#diff-djEvc3JjL21haW4vamF2YS9jb20vZ29vZ2xlL2Nsb3VkL3RlbGVwb3J0L3RlbXBsYXRlcy9ETFBUZXh0VG9CaWdRdWVyeVN0cmVhbWluZy5qYXZh) | `38.55% <ø> (ø)` | | | ... and [25 more](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | | ... and [55 files with indirect coverage changes](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1667/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform)