search

ExpediaGroup / circus-train

Circus Train is a dataset replication tool that copies Hive tables between clusters and clouds.
Apache License 2.0
86 stars 15 forks source link

out of date spring boot libraries when download libraries or build from sources #233

Open mitnitskiyas opened 2 years ago

mitnitskiyas commented 2 years ago

Hello everyone,

Used products

Linux version 5.10.47-linuxkit (root@buildkitsandbox) (gcc (Alpine 10.2.1_pre1) 10.2.1 20201203, GNU ld (GNU Binutils) 2.35.2) #1 SMP Sat Jul 3 21:51:47 UTC 2021 Jdk 1.8.0_291 Circus-train 16.4.0 Cloudera Hadoop 3.0.0-cdh6.2.1 Hive 2.1.1-cdh6.2.1

Description of the bug:

After downloading binary version circus-train-16.4.0-bin.tgz, extracting into /opt/circus/circus-train-16.4.0 setting up the variables in config files

export HIVE_LIB=/opt/cloudera/parcels/CDH/lib/hive/lib export HIVE_CONF_PATH=/opt/conf/cloudift/conf.cloudera.hive export HCAT_LIB=/opt/cloudera/parcels/CDH/lib/hive-hcatalog/etc/hcatalog

bash-4.2# ./circus-train.sh Using /opt/circus/circus-train-16.4.0 WARNING: Use "yarn jar" to launch YARN applications. SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.2.1-1.cdh6.2.1.p4184.5581308/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/circus/circus-train-16.4.0/lib/slf4j-log4j12-1.7.21.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 22/04/08 12:20:14 INFO circustrain.CircusTrain: Maven-ArtifactId=circus-train-core; Build-DateTime=2021-08-24T14:08:09Z; Build-Jdk-Spec=1.8; Created-By=Maven Jar Plugin 3.2.0; Build-Version=16.4.0; Manifest-Version=1.0; Maven-GroupId=com.hotels; ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'org.apache.logging.log4j.simplelog.StatusLogger.level' to TRACE to show Log4j2 internal initialization logging. 22/04/08 12:20:20 ERROR boot.SpringApplication: Application startup failed java.lang.NoSuchMethodError: org.apache.logging.log4j.core.config.ConfigurationFactory.getConfiguration(Lorg/apache/logging/log4j/core/config/ConfigurationSource;)Lorg/apache/logging/log4j/core/config/Configuration; at org.springframework.boot.logging.log4j2.Log4J2LoggingSystem.loadConfiguration(Log4J2LoggingSystem.java:167) at org.springframework.boot.logging.log4j2.Log4J2LoggingSystem.loadDefaults(Log4J2LoggingSystem.java:150) at org.springframework.boot.logging.AbstractLoggingSystem.initializeWithConventions(AbstractLoggingSystem.java:75) at org.springframework.boot.logging.AbstractLoggingSystem.initialize(AbstractLoggingSystem.java:50) at org.springframework.boot.logging.log4j2.Log4J2LoggingSystem.initialize(Log4J2LoggingSystem.java:140) at org.springframework.boot.logging.LoggingApplicationListener.initializeSystem(LoggingApplicationListener.java:277) at org.springframework.boot.logging.LoggingApplicationListener.initialize(LoggingApplicationListener.java:255) at org.springframework.boot.logging.LoggingApplicationListener.onApplicationEnvironmentPreparedEvent(LoggingApplicationListener.java:224) at org.springframework.boot.logging.LoggingApplicationListener.onApplicationEvent(LoggingApplicationListener.java:200) at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:166) at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:138) at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:121) at org.springframework.boot.context.event.EventPublishingRunListener.publishEvent(EventPublishingRunListener.java:111) at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:65) at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:54) at org.springframework.boot.SpringApplication.createAndRefreshContext(SpringApplication.java:329) at org.springframework.boot.SpringApplication.run(SpringApplication.java:306) at com.hotels.bdp.circustrain.CircusTrain.main(CircusTrain.java:118) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:313) at org.apache.hadoop.util.RunJar.main(RunJar.java:227) 22/04/08 12:20:20 INFO logging.ClasspathLoggingApplicationListener: Application failed to start with classpath: [file:/tmp/hadoop-unjar8402428958887069152/, file:/opt/circus/circus-train-16.4.0/lib/circus-train-all-latest.jar, file:/tmp/hadoop-unjar8402428958887069152/classes]

Manually this can be fixed by replacing manually spring-boot-1.3.8.RELEASE.jar =>spring-boot-1.5.22.RELEASE.jar and spring-core-4.2.8.RELEASE.jar=>spring-core-4.3.9.RELEASE.jar.

I think it is necessary to change the scenario while building the project.