awslabs / aws-athena-query-federation

The Amazon Athena Query Federation SDK allows you to customize Amazon Athena with your own data sources and code.
Apache License 2.0
562 stars 297 forks source link

[BUG] tools/validate_connector.sh fails with build error #2367

Open tubular400 opened 3 weeks ago

tubular400 commented 3 weeks ago

Describe the bug

I can't use tools/validate_connector.sh. I get a build failure when I run it.

To Reproduce

  1. Clone this repository.
  2. Change directory to the root of the new Git working tree.
  3. Give this command: echo Y | tools/validate_connector.sh --help

Expected behavior

I expect a help message explaining how to run tools/validate_connector.sh.

Screenshots / Exceptions / Errors

No help message is emitted. Instead, a build failure occurs.

Full output of command ``` # Run this script from any directory: # 1. Builds the maven project, if needed. # 2. Simulates an Athena query running against your connector that is deployed as a Lambda function. # # NOTE: That this test may cause a full table scan against your data source. If prompted to provide a # query predicate, doing so will avoid a full table scan. You can also opt to stop the simulated query # after the 'planning phase' so that it does not simulate process any splits. # # Use the -h or --help args to print usage information. # # Use 'yes | tools/validate_connector.sh [args]' to bypass this check. USE CAUTION # Proceeding... [INFO] Scanning for projects... [INFO] [INFO] -------------< com.amazonaws:athena-federation-sdk-tools >-------------- [INFO] Building Amazon Athena Query Federation SDK Tools 2022.47.1 [INFO] from pom.xml [INFO] --------------------------------[ jar ]--------------------------------- [WARNING] Parameter 'encoding' is unknown for plugin 'maven-checkstyle-plugin:3.6.0:check (validate)' [INFO] [INFO] --- clean:3.2.0:clean (default-clean) @ athena-federation-sdk-tools --- [INFO] [INFO] --- checkstyle:3.6.0:check (validate) @ athena-federation-sdk-tools --- [WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent! [INFO] Starting audit... Audit done. [INFO] You have 0 Checkstyle violations. [INFO] [INFO] --- jacoco:0.8.12:prepare-agent (default) @ athena-federation-sdk-tools --- [INFO] argLine set to -javaagent:/Users/polytope_8747/.m2/repository/org/jacoco/org.jacoco.agent/0.8.12/org.jacoco.agent-0.8.12-runtime.jar=destfile=/private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/target/jacoco.exec [INFO] [INFO] --- license:2.4.0:update-file-header (first) @ athena-federation-sdk-tools --- [WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent! [INFO] Adding a license repository jar:file:/Users/polytope_8747/.m2/repository/org/codehaus/mojo/license-maven-plugin/2.4.0/license-maven-plugin-2.4.0.jar!/META-INF/licenses [INFO] Will search files to update from root /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console... [INFO] Scan 5 files header done in 46.136ms. [INFO] All files are up-to-date. [INFO] [INFO] --- resources:3.3.1:resources (default-resources) @ athena-federation-sdk-tools --- [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 1 resource from src/main/resources to target/classes [INFO] [INFO] --- compiler:3.13.0:compile (default-compile) @ athena-federation-sdk-tools --- [INFO] Recompiling the module because of changed source code. [WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent! [INFO] Compiling 5 source files with javac [debug release 11] to target/classes [INFO] ------------------------------------------------------------- [WARNING] COMPILATION WARNING : [INFO] ------------------------------------------------------------- [WARNING] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/ConnectorValidator.java:[466,42] non-varargs call of varargs method with inexact argument type for last parameter; cast to java.lang.Object for a varargs call cast to java.lang.Object[] for a non-varargs call and to suppress this warning [INFO] 1 warning [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------- [ERROR] COMPILATION ERROR : [INFO] ------------------------------------------------------------- [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/ConstraintParser.java:[38,1] cannot find symbol symbol: static DEFAULT_NO_LIMIT location: class com.amazonaws.athena.connector.lambda.domain.predicate.Constraints [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[32,35] package software.amazon.awssdk.core does not exist [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[33,46] package software.amazon.awssdk.services.lambda does not exist [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[34,52] package software.amazon.awssdk.services.lambda.model does not exist [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[35,52] package software.amazon.awssdk.services.lambda.model does not exist [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[53,26] cannot find symbol symbol: class LambdaClient location: class com.amazonaws.athena.connector.validation.FederationServiceProvider [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/ConstraintParser.java:[139,103] cannot find symbol symbol: variable DEFAULT_NO_LIMIT location: class com.amazonaws.athena.connector.validation.ConstraintParser [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/ConstraintParser.java:[148,91] cannot find symbol symbol: variable DEFAULT_NO_LIMIT location: class com.amazonaws.athena.connector.validation.ConstraintParser [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[53,54] cannot find symbol symbol: variable LambdaClient location: class com.amazonaws.athena.connector.validation.FederationServiceProvider [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[70,9] cannot find symbol symbol: class InvokeRequest location: class com.amazonaws.athena.connector.validation.FederationServiceProvider [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[70,39] cannot find symbol symbol: variable InvokeRequest location: class com.amazonaws.athena.connector.validation.FederationServiceProvider [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[72,26] cannot find symbol symbol: variable SdkBytes location: class com.amazonaws.athena.connector.validation.FederationServiceProvider [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[75,9] cannot find symbol symbol: class InvokeResponse location: class com.amazonaws.athena.connector.validation.FederationServiceProvider [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/LambdaMetadataProvider.java:[137,18] constructor GetTableRequest in class com.amazonaws.athena.connector.lambda.metadata.GetTableRequest cannot be applied to given types; required: com.amazonaws.athena.connector.lambda.security.FederatedIdentity,java.lang.String,java.lang.String,com.amazonaws.athena.connector.lambda.domain.TableName found: com.amazonaws.athena.connector.lambda.security.FederatedIdentity,java.lang.String,java.lang.String,com.amazonaws.athena.connector.lambda.domain.TableName,java.util.Map reason: actual and formal argument lists differ in length [INFO] 14 errors [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.821 s [INFO] Finished at: 2024-10-31T09:15:48Z [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.13.0:compile (default-compile) on project athena-federation-sdk-tools: Compilation failure: Compilation failure: [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/ConstraintParser.java:[38,1] cannot find symbol [ERROR] symbol: static DEFAULT_NO_LIMIT [ERROR] location: class com.amazonaws.athena.connector.lambda.domain.predicate.Constraints [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[32,35] package software.amazon.awssdk.core does not exist [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[33,46] package software.amazon.awssdk.services.lambda does not exist [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[34,52] package software.amazon.awssdk.services.lambda.model does not exist [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[35,52] package software.amazon.awssdk.services.lambda.model does not exist [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[53,26] cannot find symbol [ERROR] symbol: class LambdaClient [ERROR] location: class com.amazonaws.athena.connector.validation.FederationServiceProvider [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/ConstraintParser.java:[139,103] cannot find symbol [ERROR] symbol: variable DEFAULT_NO_LIMIT [ERROR] location: class com.amazonaws.athena.connector.validation.ConstraintParser [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/ConstraintParser.java:[148,91] cannot find symbol [ERROR] symbol: variable DEFAULT_NO_LIMIT [ERROR] location: class com.amazonaws.athena.connector.validation.ConstraintParser [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[53,54] cannot find symbol [ERROR] symbol: variable LambdaClient [ERROR] location: class com.amazonaws.athena.connector.validation.FederationServiceProvider [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[70,9] cannot find symbol [ERROR] symbol: class InvokeRequest [ERROR] location: class com.amazonaws.athena.connector.validation.FederationServiceProvider [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[70,39] cannot find symbol [ERROR] symbol: variable InvokeRequest [ERROR] location: class com.amazonaws.athena.connector.validation.FederationServiceProvider [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[72,26] cannot find symbol [ERROR] symbol: variable SdkBytes [ERROR] location: class com.amazonaws.athena.connector.validation.FederationServiceProvider [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/FederationServiceProvider.java:[75,9] cannot find symbol [ERROR] symbol: class InvokeResponse [ERROR] location: class com.amazonaws.athena.connector.validation.FederationServiceProvider [ERROR] /private/tmp/aws-athena-query-federation/athena-federation-sdk-tools/src/main/java/com/amazonaws/athena/connector/validation/LambdaMetadataProvider.java:[137,18] constructor GetTableRequest in class com.amazonaws.athena.connector.lambda.metadata.GetTableRequest cannot be applied to given types; [ERROR] required: com.amazonaws.athena.connector.lambda.security.FederatedIdentity,java.lang.String,java.lang.String,com.amazonaws.athena.connector.lambda.domain.TableName [ERROR] found: com.amazonaws.athena.connector.lambda.security.FederatedIdentity,java.lang.String,java.lang.String,com.amazonaws.athena.connector.lambda.domain.TableName,java.util.Map [ERROR] reason: actual and formal argument lists differ in length [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException Error: Could not find or load main class com.amazonaws.athena.connector.validation.ConnectorValidator Caused by: java.lang.ClassNotFoundException: com.amazonaws.athena.connector.validation.ConnectorValidator ```

Connector Details (please complete the following information):

N/A

Additional context

My OS is macOS 15.0.1.

Output of type -a java:

java is /usr/bin/java

Output of java -version:

openjdk version "16.0.2" 2021-07-20
OpenJDK Runtime Environment Corretto-16.0.2.7.1 (build 16.0.2+7)
OpenJDK 64-Bit Server VM Corretto-16.0.2.7.1 (build 16.0.2+7, mixed mode, sharing)

Output of mvn --version:

Apache Maven 3.9.9 (8e8579a9e76f7d015ee5ec7bfcdc97d260186937)
Maven home: /opt/homebrew/Cellar/maven/3.9.9/libexec
Java version: 23, vendor: Homebrew, runtime: /opt/homebrew/Cellar/openjdk/23/libexec/openjdk.jdk/Contents/Home
Default locale: en_IE, platform encoding: UTF-8
OS name: "mac os x", version: "15.0.1", arch: "aarch64", family: "mac"