search

izhangzhihao / Real-time-Data-Warehouse

Real-time Data Warehouse with Apache Flink & Apache Kafka & Apache Hudi
109 stars 42 forks source link

Images under rtdw in dockerhub doesnt exists anymore? #3

Closed Soufraz closed 2 years ago

Soufraz commented 2 years ago

In docker hub i cant access these images:

rtdw/flink-base rtdw/flink-sql-client

Is it available in another account? If so, can you share the account name?

Thank you.

izhangzhihao commented 2 years ago

yes, you need build it locally by running docker compose build

Soufraz commented 2 years ago

Yeah my mistake. This is in readme and i buid it. The error is not related with account in dockerhub. But after download flink connectors, I am facing a copy error. I'll try make a copy from /opt/flink/lib/ instead jars/* to test. 

docker-compose build                                                                                             ✔ 
zookeeper uses an image, skipping
kafka uses an image, skipping
postgres uses an image, skipping
connect uses an image, skipping
jobmanager uses an image, skipping
taskmanager uses an image, skipping
elasticsearch uses an image, skipping
kibana uses an image, skipping
Building flink-base-img
Sending build context to Docker daemon  24.58kB
Step 1/6 : FROM flink:1.12.2-scala_2.12-java11
 ---> 67560b1a511f
Step 2/6 : RUN wget -P /opt/flink/lib/ https://maven.aliyun.com/repository/central/org/apache/flink/flink-json/1.12.2/flink-json-1.12.2.jar;     wget -P /opt/flink/lib/ https://maven.aliyun.com/repository/central/org/apache/flink/flink-sql-connector-kafka_2.12/1.12.2/flink-sql-connector-kafka_2.12-1.12.2.jar;     wget -P /opt/flink/lib/ https://maven.aliyun.com/repository/central/org/apache/flink/flink-sql-connector-elasticsearch7_2.12/1.12.2/flink-sql-connector-elasticsearch7_2.12-1.12.2.jar;     wget -P /opt/flink/lib/ https://maven.aliyun.com/repository/central/org/apache/flink/flink-connector-jdbc_2.12/1.12.2/flink-connector-jdbc_2.12-1.12.2.jar;     wget -P /opt/flink/lib/ https://maven.aliyun.com/repository/central/org/postgresql/postgresql/42.2.14/postgresql-42.2.14.jar;     wget -P /opt/flink/lib/ https://maven.aliyun.com/repository/central/mysql/mysql-connector-java/8.0.25/mysql-connector-java-8.0.25.jar;     wget -P /opt/flink/lib/ https://maven.aliyun.com/repository/central/org/apache/flink/flink-connector-kafka_2.12/1.12.2/flink-connector-kafka_2.12-1.12.2.jar;     wget -P /opt/flink/lib/ https://maven.aliyun.com/repository/central/org/apache/kafka/kafka-clients/2.4.1/kafka-clients-2.4.1.jar;
 ---> Using cache
 ---> 716fee56fb8a
Step 3/6 : RUN wget -P /opt/flink/lib/ https://maven.aliyun.com/repository/central/commons-logging/commons-logging/1.2/commons-logging-1.2.jar;
 ---> Using cache
 ---> 92a0b135d5f8
Step 4/6 : RUN wget -P /opt/flink/lib/ https://maven.aliyun.com/repository/central/org/apache/flink/flink-shaded-hadoop-2-uber/2.8.3-10.0/flink-shaded-hadoop-2-uber-2.8.3-10.0.jar;
 ---> Using cache
 ---> e367ec0583c8
Step 5/6 : COPY jars/* /opt/flink/lib/
COPY failed: no source files were specified
ERROR: Service 'flink-base-img' failed to build : Build failed
izhangzhihao commented 2 years ago

If there are no additional jars need to build, please remove the line COPY jars/* /opt/flink/lib/

HarjeetSinghGoldy commented 1 year ago

Getting this error while inserting data to hudi.

[INFO] Submitting SQL update statement to the cluster... [ERROR] Could not execute SQL statement. Reason: java.lang.ClassCastException: class org.codehaus.janino.CompilerFactory cannot be cast to class org.codehaus.commons.compiler.ICompilerFactory (org.codehaus.janino.CompilerFactory is in unnamed module of loader org.apache.flink.util.ChildFirstClassLoader @fd8294b; org.codehaus.commons.compiler.ICompilerFactory is in unnamed module of loader 'app')