search

aws-samples / amazon-kinesis-data-analytics-examples

Example applications in Java, Python and SQL for Kinesis Data Analytics, demonstrating sources, sinks, and operators.
MIT No Attribution
139 stars 87 forks source link

As long as java 11 or above, then fine #45

Closed yezhengli-Mr9 closed 1 year ago

yezhengli-Mr9 commented 1 year ago

As long as java 11 or above:

mvn -v
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /Users/[user]/apache-maven-3.6.3
Java version: 19.0.1, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk-19.jdk/Contents/Home
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "13.0.1", arch: "x86_64", family: "mac"

then mvn package -Dflink.version=1.15.2 is fine for this tutorial.

[INFO] Scanning for projects...
[INFO]
[INFO] -----------< com.amazonaws:aws-kinesis-analytics-java-apps >------------
[INFO] Building aws-kinesis-analytics-java-apps 1.0
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ aws-kinesis-analytics-java-apps ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/[user]/github/amazon-kinesis-data-analytics-java-examples/GettingStarted/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ aws-kinesis-analytics-java-apps ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
[INFO] Compiling 1 source file to /Users/[user]/github/amazon-kinesis-data-analytics-java-examples/GettingStarted/target/classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ aws-kinesis-analytics-java-apps ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/[user]/github/amazon-kinesis-data-analytics-java-examples/GettingStarted/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ aws-kinesis-analytics-java-apps ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ aws-kinesis-analytics-java-apps ---
[INFO] No tests to run.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ aws-kinesis-analytics-java-apps ---
[INFO] Building jar: /Users/[user]/github/amazon-kinesis-data-analytics-java-examples/GettingStarted/target/aws-kinesis-analytics-java-apps-1.0.jar
[INFO]
[INFO] --- maven-shade-plugin:3.2.1:shade (default) @ aws-kinesis-analytics-java-apps ---
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-artifact-transfer/0.10.0/maven-artifact-transfer-0.10.0.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-artifact-transfer/0.10.0/maven-artifact-transfer-0.10.0.pom (12 kB at 31 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/31/maven-shared-components-31.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/31/maven-shared-components-31.pom (5.1 kB at 75 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/7.0/asm-7.0.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/7.0/asm-7.0.pom (2.9 kB at 48 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-commons/7.0/asm-commons-7.0.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-commons/7.0/asm-commons-7.0.pom (3.7 kB at 78 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-tree/7.0/asm-tree-7.0.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-tree/7.0/asm-tree-7.0.pom (3.1 kB at 69 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-analysis/7.0/asm-analysis-7.0.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-analysis/7.0/asm-analysis-7.0.pom (3.2 kB at 67 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/vafer/jdependency/2.1.1/jdependency-2.1.1.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/vafer/jdependency/2.1.1/jdependency-2.1.1.pom (11 kB at 166 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/7.0-beta/asm-7.0-beta.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/7.0-beta/asm-7.0-beta.pom (2.9 kB at 52 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-analysis/7.0-beta/asm-analysis-7.0-beta.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-analysis/7.0-beta/asm-analysis-7.0-beta.pom (3.2 kB at 66 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-tree/7.0-beta/asm-tree-7.0-beta.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-tree/7.0-beta/asm-tree-7.0-beta.pom (3.1 kB at 60 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-commons/7.0-beta/asm-commons-7.0-beta.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-commons/7.0-beta/asm-commons-7.0-beta.pom (3.7 kB at 88 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-util/7.0-beta/asm-util-7.0-beta.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-util/7.0-beta/asm-util-7.0-beta.pom (3.7 kB at 82 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/19.0/guava-19.0.pom
Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/19.0/guava-19.0.pom (6.8 kB at 154 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/19.0/guava-parent-19.0.pom
Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/19.0/guava-parent-19.0.pom (9.9 kB at 219 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.1.0/plexus-utils-3.1.0.jar
Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/7.0/asm-7.0.jar
Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-commons/7.0/asm-commons-7.0.jar
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-artifact-transfer/0.10.0/maven-artifact-transfer-0.10.0.jar
Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-tree/7.0/asm-tree-7.0.jar
Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.1.0/plexus-utils-3.1.0.jar (262 kB at 2.7 MB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-analysis/7.0/asm-analysis-7.0.jar
Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/7.0/asm-7.0.jar (114 kB at 806 kB/s)
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-artifact-transfer/0.10.0/maven-artifact-transfer-0.10.0.jar (128 kB at 907 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-util/7.0-beta/asm-util-7.0-beta.jar
Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-tree/7.0/asm-tree-7.0.jar (50 kB at 355 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/vafer/jdependency/2.1.1/jdependency-2.1.1.jar
Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/19.0/guava-19.0.jar
Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-analysis/7.0/asm-analysis-7.0.jar (33 kB at 235 kB/s)
Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-commons/7.0/asm-commons-7.0.jar (80 kB at 515 kB/s)
Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-util/7.0-beta/asm-util-7.0-beta.jar (81 kB at 414 kB/s)
Downloaded from central: https://repo.maven.apache.org/maven2/org/vafer/jdependency/2.1.1/jdependency-2.1.1.jar (186 kB at 843 kB/s)
Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/19.0/guava-19.0.jar (2.3 MB at 6.7 MB/s)
[INFO] Including com.amazonaws:aws-kinesisanalytics-runtime:jar:1.2.0 in the shaded jar.
[INFO] Including org.apache.flink:flink-connector-kinesis:jar:1.15.2 in the shaded jar.
[INFO] Including joda-time:joda-time:jar:2.5 in the shaded jar.
[INFO] Including commons-io:commons-io:jar:2.11.0 in the shaded jar.
[INFO] Including commons-lang:commons-lang:jar:2.6 in the shaded jar.
[INFO] Including org.apache.commons:commons-lang3:jar:3.3.2 in the shaded jar.
[INFO] Including commons-logging:commons-logging:jar:1.1.3 in the shaded jar.
[INFO] Including com.fasterxml.jackson.core:jackson-core:jar:2.13.2 in the shaded jar.
[INFO] Including com.google.guava:guava:jar:29.0-jre in the shaded jar.
[INFO] Including com.google.guava:failureaccess:jar:1.0.1 in the shaded jar.
[INFO] Including com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava in the shaded jar.
[INFO] Including org.checkerframework:checker-qual:jar:2.11.1 in the shaded jar.
[INFO] Including com.google.errorprone:error_prone_annotations:jar:2.3.4 in the shaded jar.
[INFO] Including com.google.j2objc:j2objc-annotations:jar:1.3 in the shaded jar.
[INFO] Including org.apache.flink:flink-core:jar:1.15.2 in the shaded jar.
[INFO] Including org.apache.flink:flink-shaded-guava:jar:30.1.1-jre-15.0 in the shaded jar.
[INFO] Including org.apache.flink:flink-connector-base:jar:1.15.2 in the shaded jar.
[INFO] Including org.apache.flink:flink-table-common:jar:1.15.2 in the shaded jar.
[INFO] Including org.apache.flink:flink-annotations:jar:1.15.2 in the shaded jar.
[INFO] Including org.apache.flink:flink-metrics-core:jar:1.15.2 in the shaded jar.
[INFO] Including org.apache.flink:flink-shaded-asm-9:jar:9.2-15.0 in the shaded jar.
[INFO] Including com.esotericsoftware.kryo:kryo:jar:2.24.0 in the shaded jar.
[INFO] Including com.esotericsoftware.minlog:minlog:jar:1.2 in the shaded jar.
[INFO] Including commons-collections:commons-collections:jar:3.2.2 in the shaded jar.
[INFO] Including org.apache.commons:commons-compress:jar:1.21 in the shaded jar.
[INFO] Including com.fasterxml.jackson.core:jackson-annotations:jar:2.13.2 in the shaded jar.
[INFO] Including com.ibm.icu:icu4j:jar:67.1 in the shaded jar.
[INFO] Including commons-codec:commons-codec:jar:1.15 in the shaded jar.
[INFO] Including com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.13.2 in the shaded jar.
[INFO] Including com.fasterxml.jackson.core:jackson-databind:jar:2.13.2.2 in the shaded jar.
[INFO] Including org.apache.flink:flink-shaded-force-shading:jar:15.0 in the shaded jar.
[INFO] Including org.objenesis:objenesis:jar:2.1 in the shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.32 from the shaded jar.
[INFO] Excluding com.google.code.findbugs:jsr305:jar:1.3.9 from the shaded jar.
[INFO] Including org.apache.flink:flink-connector-aws-kinesis-streams:jar:1.15.2 in the shaded jar.
[INFO] Including org.apache.flink:flink-connector-aws-base:jar:1.15.2 in the shaded jar.
[INFO] Including software.amazon.awssdk:netty-nio-client:jar:2.17.247 in the shaded jar.
[INFO] Including io.netty:netty-codec-http:jar:4.1.77.Final in the shaded jar.
[INFO] Including io.netty:netty-codec-http2:jar:4.1.77.Final in the shaded jar.
[INFO] Including io.netty:netty-codec:jar:4.1.77.Final in the shaded jar.
[INFO] Including io.netty:netty-transport:jar:4.1.77.Final in the shaded jar.
[INFO] Including io.netty:netty-resolver:jar:4.1.77.Final in the shaded jar.
[INFO] Including io.netty:netty-common:jar:4.1.77.Final in the shaded jar.
[INFO] Including io.netty:netty-buffer:jar:4.1.77.Final in the shaded jar.
[INFO] Including io.netty:netty-handler:jar:4.1.77.Final in the shaded jar.
[INFO] Including io.netty:netty-transport-classes-epoll:jar:4.1.77.Final in the shaded jar.
[INFO] Including io.netty:netty-transport-native-unix-common:jar:4.1.77.Final in the shaded jar.
[INFO] Including org.reactivestreams:reactive-streams:jar:1.0.3 in the shaded jar.
[INFO] Including software.amazon.awssdk:sts:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:aws-query-protocol:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:kinesis:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:aws-cbor-protocol:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:third-party-jackson-dataformat-cbor:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:third-party-jackson-core:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:aws-json-protocol:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:json-utils:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:protocol-core:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:profiles:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:sdk-core:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:auth:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.eventstream:eventstream:jar:1.0.1 in the shaded jar.
[INFO] Including software.amazon.awssdk:http-client-spi:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:regions:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:annotations:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:utils:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:aws-core:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:metrics-spi:jar:2.17.247 in the shaded jar.
[INFO] Including software.amazon.awssdk:apache-client:jar:2.17.247 in the shaded jar.
[INFO] Including org.apache.httpcomponents:httpclient:jar:4.5.13 in the shaded jar.
[INFO] Including org.apache.httpcomponents:httpcore:jar:4.4.13 in the shaded jar.
[WARNING] Discovered module-info.class. Shading will break its strong encapsulation.
[WARNING] flink-connector-aws-base-1.15.2.jar, flink-connector-kinesis-1.15.2.jar define 2 overlapping classes:
[WARNING]   - org.apache.flink.connector.aws.table.util.AsyncClientOptionsUtils
[WARNING]   - org.apache.flink.connector.aws.table.util.AWSOptionUtils
[WARNING] flink-connector-kinesis-1.15.2.jar, flink-connector-aws-kinesis-streams-1.15.2.jar define 21 overlapping classes:
[WARNING]   - org.apache.flink.connector.kinesis.sink.KinesisStreamsException
[WARNING]   - org.apache.flink.connector.kinesis.table.KinesisPartitionKeyGeneratorFactory
[WARNING]   - org.apache.flink.connector.kinesis.table.util.KinesisStreamsConnectorOptionsUtils$KinesisProducerOptionsMapper
[WARNING]   - org.apache.flink.connector.kinesis.table.KinesisDynamicTableSinkFactory
[WARNING]   - org.apache.flink.connector.kinesis.sink.KinesisStreamsSinkBuilder
[WARNING]   - org.apache.flink.connector.kinesis.table.RandomKinesisPartitionKeyGenerator
[WARNING]   - org.apache.flink.connector.kinesis.sink.KinesisStreamsSinkElementConverter
[WARNING]   - org.apache.flink.connector.kinesis.sink.KinesisStreamsStateSerializer
[WARNING]   - org.apache.flink.connector.kinesis.sink.KinesisStreamsException$KinesisStreamsFailFastException
[WARNING]   - org.apache.flink.connector.kinesis.table.KinesisDynamicSink$KinesisDynamicTableSinkBuilder
[WARNING]   - 11 more...
[WARNING] third-party-jackson-core-2.17.247.jar, jackson-dataformat-cbor-2.13.2.jar, flink-connector-kinesis-1.15.2.jar, jackson-core-2.13.2.jar, third-party-jackson-dataformat-cbor-2.17.247.jar, jackson-databind-2.13.2.2.jar define 1 overlapping classes:
[WARNING]   - META-INF.versions.9.module-info
[WARNING] maven-shade-plugin has detected that some class files are
[WARNING] present in two or more JARs. When this happens, only one
[WARNING] single version of the class is copied to the uber jar.
[WARNING] Usually this is not harmful and you can skip these warnings,
[WARNING] otherwise try to manually exclude artifacts based on
[WARNING] mvn dependency:tree -Ddetail=true and the above output.
[WARNING] See http://maven.apache.org/plugins/maven-shade-plugin/
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /Users/[user]/github/amazon-kinesis-data-analytics-java-examples/GettingStarted/target/aws-kinesis-analytics-java-apps-1.0.jar with /Users/[user]/github/amazon-kinesis-data-analytics-java-examples/GettingStarted/target/aws-kinesis-analytics-java-apps-1.0-shaded.jar
[INFO] Dependency-reduced POM written at: /Users/[user]/github/amazon-kinesis-data-analytics-java-examples/GettingStarted/dependency-reduced-pom.xml
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  13.185 s
[INFO] Finished at: 2022-12-10T18:41:56-06:00
[INFO] ------------------------------------------------------------------------
(base) LM-SJC-11024946:GettingStarted [user]$ ls *target
aws-kinesis-analytics-java-apps-1.0.jar         maven-archiver
classes                         maven-status
generated-sources                   original-aws-kinesis-analytics-java-apps-1.0.jar