GoogleCloudPlatform / DataflowTemplates

Cloud Dataflow Google-provided templates for solving in-Cloud data tasks
https://cloud.google.com/dataflow/docs/guides/templates/provided-templates
Apache License 2.0
1.14k stars 950 forks source link

Fix annotation processor #1744

Closed Polber closed 1 month ago

Polber commented 1 month ago

1677 changed the way that maven-compiler-plugin compiles annotation processors from being implicit to explicitly defined. The definition for @AutoValue annotations was added, but not @AutoService.

This caused the CommonTemplateJvmInitializer to not get compiled into the template jars with the @AutoService handler, which resulted in Beam not calling the necessary Jvm initialization methods for certain templates to work properly. https://github.com/GoogleCloudPlatform/DataflowTemplates/blob/0d01accc1cfa29b762a764bc226496bfde25165c/v1/src/main/java/com/google/cloud/teleport/templates/common/CommonTemplateJvmInitializer.java#L40-L44

This was namely an issue with JDBC templates for users that rely on the --extraFilesToStage parameter to copy files from GCS to the extra_files directory on the Dataflow worker.

This PR adds the definition for the @AutoService annotation processor to affected modules.

codecov[bot] commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 42.31%. Comparing base (5be5817) to head (10bdd2a).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1744 +/- ## ========================================= Coverage 42.31% 42.31% Complexity 3180 3180 ========================================= Files 794 794 Lines 46244 46244 Branches 4951 4951 ========================================= Hits 19568 19568 Misses 25077 25077 Partials 1599 1599 ``` | [Components](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1744/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | Coverage Δ | | |---|---|---| | [spanner-templates](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1744/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `63.69% <ø> (ø)` | | | [spanner-import-export](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1744/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `64.42% <ø> (ø)` | | | [spanner-live-forward-migration](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1744/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `74.97% <ø> (ø)` | | | [spanner-live-reverse-replication](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1744/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `51.84% <ø> (ø)` | | | [spanner-bulk-migration](https://app.codecov.io/gh/GoogleCloudPlatform/DataflowTemplates/pull/1744/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleCloudPlatform) | `82.79% <ø> (ø)` | |
Polber commented 1 month ago

Test failure unrelated