GoogleCloudPlatform / DataflowTemplates

Cloud Dataflow Google-provided templates for solving in-Cloud data tasks
https://cloud.google.com/dataflow/docs/guides/templates/provided-templates
Apache License 2.0
1.14k stars 951 forks source link

[Bug]: Compilation Error with MongoDB to BigQuery #1793

Open wazi55 opened 1 month ago

wazi55 commented 1 month ago

Related Template(s)

MongoDbToBigQuery & MongoDbToBigQueryCDC

Template Version

Don't know

What happened?

While trying to make a contribution to allow mongodb to bigquery to support Json in a USER_OPTION=FLATTEN, I cannot seem to compile the code successfuly

To reproduce:

export PROJECT="project"
export BUCKET_NAME="bucket"

mvn clean package -PtemplatesStage  \
-DskipTests \
-DprojectId="$PROJECT" \
-DbucketName="$BUCKET_NAME" \
-DstagePrefix="templates" \
-DtemplateName="MongoDB_to_BigQuery" \
-f v2/mongodb-to-googlecloud

Relevant log output

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.11.0:compile (default-compile) on project mongodb-to-googlecloud: Compilation failure
[ERROR] /Users/wazi/Downloads/DataflowTemplates/v2/mongodb-to-googlecloud/src/main/java/com/google/cloud/teleport/v2/mongodb/options/MongoDbToBigQueryOptions.java:[136,23] cannot find symbol
[ERROR]   symbol:   class JavascriptUdfFile
[ERROR]   location: class com.google.cloud.teleport.metadata.TemplateParameter
[ERROR] 
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
wazi55 commented 1 month ago

It does seem like the JavascriptUdfFile interface is in the right file DataflowTemplates/metadata/src/main/java/com/google/cloud/teleport/metadata/TemplateParameter.java