search

odpi / egeria

Egeria core
https://egeria-project.org
Apache License 2.0
809 stars 261 forks source link

add repository connector jar with hadoop as dependency, issue for Conflicting Logger modules. LoggerFactory is not a Logback LoggerContext but Logback is on the classpath #2934

Closed Shirong-Liang closed 4 years ago

Shirong-Liang commented 4 years ago

trying to add a new repository connector jar in the calsspath with -Dloader.path, the reposotory connector added has hive-metastore and hadoop-common as dependencies since want to import hivemetastoreclient, for example in the pom.xml: and in the dependency hierarchy it has log4j-slf4j 

<dependency>
  <groupId>org.apache.hive</groupId>
  <artifactId>hive-metastore</artifactId>
  <version>2.3.4</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-common -->
<dependency>
  <groupId>org.apache.hadoop</groupId>
  <artifactId>hadoop-common</artifactId>
  <version>3.1.2</version>
  <scope>provided</scope>
</dependency>

since in egeria jar it has the logback as logger dependencies, therefore it have problem when starting the egeria OMAG: at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:597) Caused by: java.lang.IllegalArgumentException: LoggerFactory is not a Logback LoggerContext but Logback is on the classpath. Either remove Logback or the competing implementation (class org.apache.logging.slf4j.Log4jLoggerFactory loaded from file:/lib/adapter-1.5-SNAPSHOT-jar-with-dependencies.jar). If you are using WebLogic you will need to add 'org.slf4j' to prefer-application-packages in WEB-INF/weblogic.xml: org.apache.logging.slf4j.Log4jLoggerFactory at org.springframework.util.Assert.instanceCheckFailed(Assert.java:696)

it seems we could not just choose a preferred logger since these jars are having different logger usage and seems we could not just exclude one of them, is there any suggestion or is this a known issue and we are in the wrong path? comments are welcome please leave any thoughts! thanks!

planetf1 commented 4 years ago

Hi. It's a good place to ask....

Most of the egeria modules that do logging will depend on org.slf4j:slf4j-api This depends on an implementation being provided at runtime - it doesn't have to be log4J - there are choices

In the server chassis we do add a dependency of ch.qos.logback:logback-classic which provides an implementation at runtime. This is generally what we do for anything with a main . In some test cases we use slf4j-simple instead - but this is not included in the main runtime code.

It looks as if in your case the hadoop environment is pulling in a log4J requirement. I expect we're always going to be exposed to differences in dependencies when additional libraries are added into the classpath - but it is an important use case, so we should understand/document some working approaches, so I'm very interested in your question and ideas.

Could you elaborate on

it seems we could not just choose a preferred logger since these jars are having different logger usage and seems we could not just exclude one of them,

I didn't quite understand your suggestion?

it may help if you could do a 'mvn dependency:tree' for your module? and/or understand the classes you are pulling in with your new connector?

I'm not a spring expert - but I can see the challenge is figuring out how to pull in only one implementation, and in particular making that call at runtime (as opposed to altering the egeria poms). Could you exclude the conflict when you build your connector?

planetf1 commented 4 years ago

My hope is perhaps when building the connector you can use some 'excludes' when specifying your dependency to remove any other logging implementations. Then only one LoggerFactory will be found at runtime.

Is your code on github & easily buildable? I'd be happy to take a look at some ideas also?

Shirong-Liang commented 4 years ago

Hi, Thanks so much for the reply!

for the suggestion I mean that it seems to me the log from both Egeria and hivemeastore/Hadoop jar are important and seems just exclude one of them is not a good idea.

sure I will push a simple version to git and follow up with you, and yes I have tried that exclude the slf4j-log4j12 from Hadoop and hive dependencies, that does get rid of this message, however it seems there are requestLog Writer in jetty server needs this components: 2020-04-16 00:56:19.897 ERROR 20734 --- [ main] o.s.boot.SpringApplication : Application run failed

java.lang.NoClassDefFoundError: org/eclipse/jetty/server/RequestLog$Writer at java.lang.Class.getDeclaredConstructors0(Native Method) ~[na:1.8.0_242] at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671) ~[na:1.8.0_242] at java.lang.Class.getDeclaredConstructors(Class.java:2020) ~[na:1.8.0_242] at org.springframework.boot.context.properties.ConfigurationPropertiesBindConstructorProvider.findConstructorBindingAnnotatedConstructor(ConfigurationPropertiesBindConstructorProvider.java:62) ~[spring-boot-2.2.5.RELEASE.jar!/:2.2.5.RELEASE] at org.springframework.boot.context.properties.ConfigurationPropertiesBindConstructorProvider.getBindConstructor(ConfigurationPropertiesBindConstructorProvider.java:48) ~[spring-boot-2.2.5.RELEASE.jar!/:2.2.5.RELEASE] at org.springframework.boot.context.properties.ConfigurationPropertiesBean$BindMethod.forType(ConfigurationPropertiesBean.java:311) ~[spring-boot-2.2.5.RELEASE.jar!/:2.2.5.RELEASE] at org.springframework.boot.context.properties.ConfigurationPropertiesBeanDefinitionValidator.validate(ConfigurationPropertiesBeanDefinitionValidator.java:63) ~[spring-boot-2.2.5.RELEASE.jar!/:2.2.5.RELEASE] at org.springframework.boot.context.properties.ConfigurationPropertiesBeanDefinitionValidator.postProcessBeanFactory(ConfigurationPropertiesBeanDefinitionValidator.java:45) ~[spring-boot-2.2.5.RELEASE.jar!/:2.2.5.RELEASE] at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:286) ~[spring-context-5.2.4.RELEASE.jar!/:5.2.4.RELEASE] at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:174) ~[spring-context-5.2.4.RELEASE.jar!/:5.2.4.RELEASE] at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:706) ~[spring-context-5.2.4.RELEASE.jar!/:5.2.4.RELEASE] at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:532) ~[spring-context-5.2.4.RELEASE.jar!/:5.2.4.RELEASE] at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141) ~[spring-boot-2.2.5.RELEASE.jar!/:2.2.5.RELEASE] at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) [spring-boot-2.2.5.RELEASE.jar!/:2.2.5.RELEASE] at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) [spring-boot-2.2.5.RELEASE.jar!/:2.2.5.RELEASE] at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) [spring-boot-2.2.5.RELEASE.jar!/:2.2.5.RELEASE] at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) [spring-boot-2.2.5.RELEASE.jar!/:2.2.5.RELEASE] at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215) [spring-boot-2.2.5.RELEASE.jar!/:2.2.5.RELEASE] at org.odpi.openmetadata.serverchassis.springboot.OMAGServerPlatform.main(OMAGServerPlatform.java:71) [classes!/:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_242] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_242] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_242] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_242] at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [server-chassis-spring-1.6.jar:na] at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [server-chassis-spring-1.6.jar:na] at org.springframework.boot.loader.Launcher.launch(Launcher.java:51) [server-chassis-spring-1.6.jar:na] at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:597) [server-chassis-spring-1.6.jar:na] Caused by: java.lang.ClassNotFoundException: org.eclipse.jetty.server.RequestLog$Writer at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[na:1.8.0_242] at java.lang.ClassLoader.loadClass(ClassLoader.java:419) ~[na:1.8.0_242] at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:92) ~[server-chassis-spring-1.6.jar:na] at java.lang.ClassLoader.loadClass(ClassLoader.java:352) ~[na:1.8.0_242] ... 27 common frames omitted

Shirong-Liang commented 4 years ago

Hi Nigel,

I just created a branch with the hadoop-common and hive-mstastore added as a dependencies in pom.xml, there is the branch https://github.com/Shirong-Liang/egeria-connector-hms-test/tree/sr/init

after build the adapter jar in this branch and put it under the -Dloader.path to start the Egeria platform will have the "oggerFactory is not a Logback LoggerContext but Logback is on the classpath" exception. if exclude the slf4j-log4j12 will cause the "NoClassDefFoundError: org/eclipse/jetty/server/RequestLog$Writer" exception.

planetf1 commented 4 years ago

I found I needed to build with an older jdk (I used Java 8) to avoid an issue with using 'tools.jar' (which is gone in later versions). Even then though I hit a compile error.

So perhaps let's just work off the errors.

I'm not experienced with hadoop libraries.. and this also gets into jetty & spring boot .. Can you get the output of 'mvn dependency:tree' of your module?

Shirong-Liang commented 4 years ago

Hi, I am also use java8: java -version openjdk version "1.8.0_242"

here is the mvn dependency:tree output

[INFO] org.odpi.egeria:egeria-connector-hms:pom:1.5-SNAPSHOT [INFO] +- junit:junit:jar:3.8.1:test​ [INFO] +- org.odpi.egeria:repository-services-apis:jar:1.6:compile​ [INFO] | +- org.slf4j:slf4j-api:jar:1.7.30:compile​ [INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.10.3:compile​ [INFO] | | - com.fasterxml.jackson.core:jackson-core:jar:2.10.3:compile​ [INFO] | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.10.3:compile​ [INFO] | +- org.odpi.egeria:audit-log-framework:jar:1.6:compile​ [INFO] | - org.apache.commons:commons-collections4:jar:4.4:compile​ [INFO] +- org.odpi.egeria:open-connector-framework:jar:1.6:compile​ [INFO] +- org.apache.hive:hive-metastore:jar:2.3.4:compile​ [INFO] | +- org.apache.hive:hive-serde:jar:2.3.4:compile​ [INFO] | | +- org.apache.hive:hive-common:jar:2.3.4:compile​ [INFO] | | | +- org.apache.hive:hive-storage-api:jar:2.4.0:compile​ [INFO] | | | +- org.apache.orc:orc-core:jar:1.3.3:compile​ [INFO] | | | | - io.airlift:aircompressor:jar:0.3:compile​ [INFO] | | | +- org.eclipse.jetty.aggregate:jetty-all:jar:7.6.0.v20120127:compile​ [INFO] | | | | +- org.apache.geronimo.specs:geronimo-jta_1.1_spec:jar:1.1.1:compile​ [INFO] | | | | +- javax.mail:mail:jar:1.4.1:compile​ [INFO] | | | | +- javax.activation:activation:jar:1.1:compile​ [INFO] | | | | +- org.apache.geronimo.specs:geronimo-jaspic_1.0_spec:jar:1.0:compile​ [INFO] | | | | +- org.apache.geronimo.specs:geronimo-annotation_1.0_spec:jar:1.1.1:compile​ [INFO] | | | | - asm:asm-commons:jar:3.1:compile​ [INFO] | | | | - asm:asm-tree:jar:3.1:compile​ [INFO] | | | | - asm:asm:jar:3.1:compile​ [INFO] | | | +- org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016:compile​ [INFO] | | | +- joda-time:joda-time:jar:2.8.1:compile​ [INFO] | | | +- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:compile​ [INFO] | | | | +- org.apache.logging.log4j:log4j-api:jar:2.6.2:compile​ [INFO] | | | | - org.apache.logging.log4j:log4j-core:jar:2.6.2:compile​ [INFO] | | | +- org.apache.logging.log4j:log4j-web:jar:2.6.2:compile​ [INFO] | | | +- org.apache.logging.log4j:log4j-slf4j-impl:jar:2.6.2:compile​ [INFO] | | | +- org.apache.ant:ant:jar:1.9.1:compile​ [INFO] | | | | - org.apache.ant:ant-launcher:jar:1.9.1:compile​ [INFO] | | | +- com.tdunning:json:jar:1.8:compile​ [INFO] | | | +- io.dropwizard.metrics:metrics-jvm:jar:3.1.0:compile​ [INFO] | | | +- io.dropwizard.metrics:metrics-json:jar:3.1.0:compile​ [INFO] | | | - com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter:jar:0.1.2:compile​ [INFO] | | +- org.apache.hive:hive-service-rpc:jar:2.3.4:compile​ [INFO] | | | +- tomcat:jasper-compiler:jar:5.5.23:compile​ [INFO] | | | | +- javax.servlet:jsp-api:jar:2.0:compile​ [INFO] | | | | - ant:ant:jar:1.6.5:compile​ [INFO] | | | - tomcat:jasper-runtime:jar:5.5.23:compile​ [INFO] | | | +- javax.servlet:servlet-api:jar:2.4:compile​ [INFO] | | | - commons-el:commons-el:jar:1.0:compile​ [INFO] | | +- net.sf.opencsv:opencsv:jar:2.3:compile​ [INFO] | | - org.apache.parquet:parquet-hadoop-bundle:jar:1.8.1:compile​ [INFO] | +- org.apache.hive:hive-shims:jar:2.3.4:compile​ [INFO] | | +- org.apache.hive.shims:hive-shims-common:jar:2.3.4:compile​ [INFO] | | +- org.apache.hive.shims:hive-shims-0.23:jar:2.3.4:runtime​ [INFO] | | | - org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.7.2:runtime​ [INFO] | | | +- com.google.inject.extensions:guice-servlet:jar:3.0:runtime​ [INFO] | | | +- com.sun.jersey.contribs:jersey-guice:jar:1.9:runtime​ [INFO] | | | +- org.apache.hadoop:hadoop-yarn-api:jar:2.7.2:compile​ [INFO] | | | +- com.sun.jersey:jersey-client:jar:1.9:runtime​ [INFO] | | | +- org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.2:runtime​ [INFO] | | | +- org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:2.7.2:runtime​ [INFO] | | | +- org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.7.2:runtime​ [INFO] | | | | - org.mortbay.jetty:jetty:jar:6.1.26:runtime​ [INFO] | | | +- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:runtime​ [INFO] | | | - org.apache.zookeeper:zookeeper:test-jar:tests:3.4.6:runtime​ [INFO] | | - org.apache.hive.shims:hive-shims-scheduler:jar:2.3.4:runtime​ [INFO] | +- javolution:javolution:jar:5.5.1:compile​ [INFO] | +- com.google.guava:guava:jar:14.0.1:compile​ [INFO] | +- com.google.protobuf:protobuf-java:jar:2.5.0:compile​ [INFO] | +- org.apache.hbase:hbase-client:jar:1.1.1:compile​ [INFO] | | +- org.apache.hbase:hbase-annotations:jar:1.1.1:compile​ [INFO] | | +- org.apache.hbase:hbase-common:jar:1.1.1:compile​ [INFO] | | | - org.mortbay.jetty:jetty-util:jar:6.1.26:compile​ [INFO] | | +- org.apache.hbase:hbase-protocol:jar:1.1.1:compile​ [INFO] | | +- io.netty:netty-all:jar:4.0.23.Final:compile​ [INFO] | | +- org.apache.htrace:htrace-core:jar:3.1.0-incubating:compile​ [INFO] | | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile​ [INFO] | | +- org.jruby.jcodings:jcodings:jar:1.0.8:compile​ [INFO] | | +- org.jruby.joni:joni:jar:2.1.2:compile​ [INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.5.1:compile​ [INFO] | | | - org.apache.hadoop:hadoop-yarn-common:jar:2.5.1:compile​ [INFO] | | - com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1:compile​ [INFO] | +- com.jolbox:bonecp:jar:0.8.0.RELEASE:compile​ [INFO] | +- com.zaxxer:HikariCP:jar:2.5.1:compile​ [INFO] | +- commons-cli:commons-cli:jar:1.2:compile​ [INFO] | +- commons-lang:commons-lang:jar:2.6:compile​ [INFO] | +- org.apache.derby:derby:jar:10.10.2.0:compile​ [INFO] | +- org.datanucleus:datanucleus-api-jdo:jar:4.2.4:compile​ [INFO] | +- org.datanucleus:datanucleus-core:jar:4.1.17:compile​ [INFO] | +- org.datanucleus:datanucleus-rdbms:jar:4.1.19:compile​ [INFO] | +- commons-pool:commons-pool:jar:1.5.4:compile​ [INFO] | +- commons-dbcp:commons-dbcp:jar:1.4:compile​ [INFO] | +- javax.jdo:jdo-api:jar:3.0.1:compile​ [INFO] | | - javax.transaction:jta:jar:1.1:compile​ [INFO] | +- org.datanucleus:javax.jdo:jar:3.2.0-m3:compile​ [INFO] | | - javax.transaction:transaction-api:jar:1.1:compile​ [INFO] | +- org.antlr:antlr-runtime:jar:3.5.2:compile​ [INFO] | +- org.apache.thrift:libfb303:jar:0.9.3:compile​ [INFO] | +- org.apache.thrift:libthrift:jar:0.9.3:compile​ [INFO] | | - org.apache.httpcomponents:httpcore:jar:4.4.1:compile​ [INFO] | +- co.cask.tephra:tephra-api:jar:0.6.0:compile​ [INFO] | +- co.cask.tephra:tephra-core:jar:0.6.0:compile​ [INFO] | | +- com.google.inject:guice:jar:3.0:compile​ [INFO] | | | +- javax.inject:javax.inject:jar:1:compile​ [INFO] | | | - aopalliance:aopalliance:jar:1.0:compile​ [INFO] | | +- com.google.inject.extensions:guice-assistedinject:jar:3.0:compile​ [INFO] | | +- it.unimi.dsi:fastutil:jar:6.5.6:compile​ [INFO] | | +- org.apache.twill:twill-common:jar:0.6.0-incubating:compile​ [INFO] | | +- org.apache.twill:twill-core:jar:0.6.0-incubating:compile​ [INFO] | | | - org.apache.twill:twill-api:jar:0.6.0-incubating:compile​ [INFO] | | +- org.apache.twill:twill-discovery-api:jar:0.6.0-incubating:compile​ [INFO] | | +- org.apache.twill:twill-discovery-core:jar:0.6.0-incubating:compile​ [INFO] | | +- org.apache.twill:twill-zookeeper:jar:0.6.0-incubating:compile​ [INFO] | | - io.dropwizard.metrics:metrics-core:jar:3.1.0:compile​ [INFO] | - co.cask.tephra:tephra-hbase-compat-1.0:jar:0.6.0:compile​ [INFO] - org.apache.hadoop:hadoop-common:jar:3.1.2:provided​ [INFO] +- org.apache.hadoop:hadoop-annotations:jar:3.1.2:compile​ [INFO] | - jdk.tools:jdk.tools:jar:1.8:system​ [INFO] +- org.apache.commons:commons-math3:jar:3.1.1:provided​ [INFO] +- org.apache.httpcomponents:httpclient:jar:4.5.2:compile​ [INFO] +- commons-codec:commons-codec:jar:1.11:compile​ [INFO] +- commons-io:commons-io:jar:2.5:compile​ [INFO] +- commons-net:commons-net:jar:3.6:provided​ [INFO] +- commons-collections:commons-collections:jar:3.2.2:compile​ [INFO] +- javax.servlet:javax.servlet-api:jar:3.1.0:provided​ [INFO] +- org.eclipse.jetty:jetty-server:jar:9.3.24.v20180605:provided​ [INFO] | +- org.eclipse.jetty:jetty-http:jar:9.3.24.v20180605:provided​ [INFO] | - org.eclipse.jetty:jetty-io:jar:9.3.24.v20180605:provided​ [INFO] +- org.eclipse.jetty:jetty-util:jar:9.3.24.v20180605:provided​ [INFO] +- org.eclipse.jetty:jetty-servlet:jar:9.3.24.v20180605:provided​ [INFO] | - org.eclipse.jetty:jetty-security:jar:9.3.24.v20180605:provided​ [INFO] +- org.eclipse.jetty:jetty-webapp:jar:9.3.24.v20180605:provided​ [INFO] | - org.eclipse.jetty:jetty-xml:jar:9.3.24.v20180605:provided​ [INFO] +- javax.servlet.jsp:jsp-api:jar:2.1:provided​ [INFO] +- com.sun.jersey:jersey-core:jar:1.19:runtime​ [INFO] | - javax.ws.rs:jsr311-api:jar:1.1.1:runtime​ [INFO] +- com.sun.jersey:jersey-servlet:jar:1.19:provided​ [INFO] +- com.sun.jersey:jersey-json:jar:1.19:runtime​ [INFO] | +- org.codehaus.jettison:jettison:jar:1.1:runtime​ [INFO] | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:runtime​ [INFO] | | - javax.xml.bind:jaxb-api:jar:2.2.2:compile​ [INFO] | | - javax.xml.stream:stax-api:jar:1.0-2:compile​ [INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.2:compile​ [INFO] | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.2:runtime​ [INFO] | - org.codehaus.jackson:jackson-xc:jar:1.9.2:runtime​ [INFO] +- com.sun.jersey:jersey-server:jar:1.19:runtime​ [INFO] +- commons-logging:commons-logging:jar:1.1.3:compile​ [INFO] +- log4j:log4j:jar:1.2.17:compile​ [INFO] +- commons-beanutils:commons-beanutils:jar:1.9.3:provided​ [INFO] +- org.apache.commons:commons-configuration2:jar:2.1.1:provided​ [INFO] +- org.apache.commons:commons-lang3:jar:3.4:compile​ [INFO] +- org.slf4j:slf4j-log4j12:jar:1.7.25:compile​ [INFO] +- org.apache.avro:avro:jar:1.7.7:compile​ [INFO] | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile​ [INFO] | - org.xerial.snappy:snappy-java:jar:1.0.5:compile​ [INFO] +- com.google.re2j:re2j:jar:1.1:provided​ [INFO] +- com.google.code.gson:gson:jar:2.2.4:compile​ [INFO] +- org.apache.hadoop:hadoop-auth:jar:3.1.2:compile​ [INFO] | +- com.nimbusds:nimbus-jose-jwt:jar:4.41.1:compile​ [INFO] | | - com.github.stephenc.jcip:jcip-annotations:jar:1.0-1:compile​ [INFO] | +- net.minidev:json-smart:jar:2.3:compile​ [INFO] | | - net.minidev:accessors-smart:jar:1.2:compile​ [INFO] | | - org.ow2.asm:asm:jar:5.0.4:compile​ [INFO] | - org.apache.curator:curator-framework:jar:2.13.0:compile​ [INFO] +- com.jcraft:jsch:jar:0.1.54:provided​ [INFO] +- org.apache.curator:curator-client:jar:2.13.0:compile​ [INFO] +- org.apache.curator:curator-recipes:jar:2.13.0:provided​ [INFO] +- com.google.code.findbugs:jsr305:jar:3.0.0:compile​ [INFO] +- org.apache.htrace:htrace-core4:jar:4.1.0-incubating:provided​ [INFO] +- org.apache.zookeeper:zookeeper:jar:3.4.13:compile​ [INFO] | +- jline:jline:jar:0.9.94:compile​ [INFO] | +- org.apache.yetus:audience-annotations:jar:0.5.0:compile​ [INFO] | - io.netty:netty:jar:3.10.6.Final:compile​ [INFO] +- org.apache.commons:commons-compress:jar:1.18:compile​ [INFO] +- org.apache.kerby:kerb-simplekdc:jar:1.0.1:compile​ [INFO] | +- org.apache.kerby:kerb-client:jar:1.0.1:compile​ [INFO] | | +- org.apache.kerby:kerby-config:jar:1.0.1:compile​ [INFO] | | +- org.apache.kerby:kerb-core:jar:1.0.1:compile​ [INFO] | | | - org.apache.kerby:kerby-pkix:jar:1.0.1:compile​ [INFO] | | | +- org.apache.kerby:kerby-asn1:jar:1.0.1:compile​ [INFO] | | | - org.apache.kerby:kerby-util:jar:1.0.1:compile​ [INFO] | | +- org.apache.kerby:kerb-common:jar:1.0.1:compile​ [INFO] | | | - org.apache.kerby:kerb-crypto:jar:1.0.1:compile​ [INFO] | | +- org.apache.kerby:kerb-util:jar:1.0.1:compile​ [INFO] | | - org.apache.kerby:token-provider:jar:1.0.1:compile​ [INFO] | - org.apache.kerby:kerb-admin:jar:1.0.1:compile​ [INFO] | +- org.apache.kerby:kerb-server:jar:1.0.1:compile​ [INFO] | | - org.apache.kerby:kerb-identity:jar:1.0.1:compile​ [INFO] | - org.apache.kerby:kerby-xdr:jar:1.0.1:compile​ [INFO] +- org.codehaus.woodstox:stax2-api:jar:3.1.4:provided​ [INFO] - com.fasterxml.woodstox:woodstox-core:jar:5.0.3:provided​ [INFO]​ [INFO] ----------------------< org.odpi.egeria:adapter >-----------------------​ [INFO] Building adapter 1.5-SNAPSHOT [2/2]​ [INFO] --------------------------------[ jar ]---------------------------------​ [INFO]​ [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ adapter ---​ [INFO] org.odpi.egeria:adapter:jar:1.5-SNAPSHOT​ [INFO] +- junit:junit:jar:3.8.1:test​ [INFO] +- org.odpi.egeria:repository-services-apis:jar:1.6:compile​ [INFO] | +- org.slf4j:slf4j-api:jar:1.7.30:compile​ [INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.10.3:compile​ [INFO] | | - com.fasterxml.jackson.core:jackson-core:jar:2.10.3:compile​ [INFO] | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.10.3:compile​ [INFO] | +- org.odpi.egeria:audit-log-framework:jar:1.6:compile​ [INFO] | - org.apache.commons:commons-collections4:jar:4.4:compile​ [INFO] +- org.odpi.egeria:open-connector-framework:jar:1.6:compile​ [INFO] +- org.apache.hive:hive-metastore:jar:2.3.4:compile​ [INFO] | +- org.apache.hive:hive-serde:jar:2.3.4:compile​ [INFO] | | +- org.apache.hive:hive-common:jar:2.3.4:compile​ [INFO] | | | +- org.apache.hive:hive-storage-api:jar:2.4.0:compile​ [INFO] | | | +- org.apache.orc:orc-core:jar:1.3.3:compile​ [INFO] | | | | - io.airlift:aircompressor:jar:0.3:compile​ [INFO] | | | +- org.eclipse.jetty.aggregate:jetty-all:jar:7.6.0.v20120127:compile​ [INFO] | | | | +- org.apache.geronimo.specs:geronimo-jta_1.1_spec:jar:1.1.1:compile​ [INFO] | | | | +- javax.mail:mail:jar:1.4.1:compile​ [INFO] | | | | +- javax.activation:activation:jar:1.1:compile​ [INFO] | | | | +- org.apache.geronimo.specs:geronimo-jaspic_1.0_spec:jar:1.0:compile​ [INFO] | | | | +- org.apache.geronimo.specs:geronimo-annotation_1.0_spec:jar:1.1.1:compile​ [INFO] | | | | - asm:asm-commons:jar:3.1:compile​ [INFO] | | | | - asm:asm-tree:jar:3.1:compile​ [INFO] | | | | - asm:asm:jar:3.1:compile​ [INFO] | | | +- org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016:compile​ [INFO] | | | +- joda-time:joda-time:jar:2.8.1:compile​ [INFO] | | | +- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:compile​ [INFO] | | | | +- org.apache.logging.log4j:log4j-api:jar:2.6.2:compile​ [INFO] | | | | - org.apache.logging.log4j:log4j-core:jar:2.6.2:compile​ [INFO] | | | +- org.apache.logging.log4j:log4j-web:jar:2.6.2:compile​ [INFO] | | | +- org.apache.logging.log4j:log4j-slf4j-impl:jar:2.6.2:compile​ [INFO] | | | +- org.apache.ant:ant:jar:1.9.1:compile​ [INFO] | | | | - org.apache.ant:ant-launcher:jar:1.9.1:compile​ [INFO] | | | +- com.tdunning:json:jar:1.8:compile​ [INFO] | | | +- io.dropwizard.metrics:metrics-jvm:jar:3.1.0:compile​ [INFO] | | | +- io.dropwizard.metrics:metrics-json:jar:3.1.0:compile​ [INFO] | | | - com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter:jar:0.1.2:compile​ [INFO] | | +- org.apache.hive:hive-service-rpc:jar:2.3.4:compile​ [INFO] | | | +- tomcat:jasper-compiler:jar:5.5.23:compile​ [INFO] | | | | +- javax.servlet:jsp-api:jar:2.0:compile​ [INFO] | | | | - ant:ant:jar:1.6.5:compile​ [INFO] | | | - tomcat:jasper-runtime:jar:5.5.23:compile​ [INFO] | | | +- javax.servlet:servlet-api:jar:2.4:compile​ [INFO] | | | - commons-el:commons-el:jar:1.0:compile​ [INFO] | | +- net.sf.opencsv:opencsv:jar:2.3:compile​ [INFO] | | - org.apache.parquet:parquet-hadoop-bundle:jar:1.8.1:compile​ [INFO] | +- org.apache.hive:hive-shims:jar:2.3.4:compile​ [INFO] | | +- org.apache.hive.shims:hive-shims-common:jar:2.3.4:compile​ [INFO] | | +- org.apache.hive.shims:hive-shims-0.23:jar:2.3.4:runtime​ [INFO] | | | - org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.7.2:runtime​ [INFO] | | | +- com.google.inject.extensions:guice-servlet:jar:3.0:runtime​ [INFO] | | | +- com.sun.jersey.contribs:jersey-guice:jar:1.9:runtime​ [INFO] | | | +- org.apache.hadoop:hadoop-yarn-api:jar:2.7.2:compile​ [INFO] | | | +- com.sun.jersey:jersey-client:jar:1.9:runtime​ [INFO] | | | +- org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.2:runtime​ [INFO] | | | +- org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:2.7.2:runtime​ [INFO] | | | +- org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.7.2:runtime​ [INFO] | | | | - org.mortbay.jetty:jetty:jar:6.1.26:runtime​ [INFO] | | | +- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:runtime​ [INFO] | | | - org.apache.zookeeper:zookeeper:test-jar:tests:3.4.6:runtime​ [INFO] | | - org.apache.hive.shims:hive-shims-scheduler:jar:2.3.4:runtime​ [INFO] | +- javolution:javolution:jar:5.5.1:compile​ [INFO] | +- com.google.guava:guava:jar:14.0.1:compile​ [INFO] | +- com.google.protobuf:protobuf-java:jar:2.5.0:compile​ [INFO] | +- org.apache.hbase:hbase-client:jar:1.1.1:compile​ [INFO] | | +- org.apache.hbase:hbase-annotations:jar:1.1.1:compile​ [INFO] | | +- org.apache.hbase:hbase-common:jar:1.1.1:compile​ [INFO] | | | - org.mortbay.jetty:jetty-util:jar:6.1.26:compile​ [INFO] | | +- org.apache.hbase:hbase-protocol:jar:1.1.1:compile​ [INFO] | | +- io.netty:netty-all:jar:4.0.23.Final:compile​ [INFO] | | +- org.apache.htrace:htrace-core:jar:3.1.0-incubating:compile​ [INFO] | | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile​ [INFO] | | +- org.jruby.jcodings:jcodings:jar:1.0.8:compile​ [INFO] | | +- org.jruby.joni:joni:jar:2.1.2:compile​ [INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.5.1:compile​ [INFO] | | | - org.apache.hadoop:hadoop-yarn-common:jar:2.5.1:compile​ [INFO] | | - com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1:compile​ [INFO] | +- com.jolbox:bonecp:jar:0.8.0.RELEASE:compile​ [INFO] | +- com.zaxxer:HikariCP:jar:2.5.1:compile​ [INFO] | +- commons-cli:commons-cli:jar:1.2:compile​ [INFO] | +- commons-lang:commons-lang:jar:2.6:compile​ [INFO] | +- org.apache.derby:derby:jar:10.10.2.0:compile​ [INFO] | +- org.datanucleus:datanucleus-api-jdo:jar:4.2.4:compile​ [INFO] | +- org.datanucleus:datanucleus-core:jar:4.1.17:compile​ [INFO] | +- org.datanucleus:datanucleus-rdbms:jar:4.1.19:compile​ [INFO] | +- commons-pool:commons-pool:jar:1.5.4:compile​ [INFO] | +- commons-dbcp:commons-dbcp:jar:1.4:compile​ [INFO] | +- javax.jdo:jdo-api:jar:3.0.1:compile​ [INFO] | | - javax.transaction:jta:jar:1.1:compile​ [INFO] | +- org.datanucleus:javax.jdo:jar:3.2.0-m3:compile​ [INFO] | | - javax.transaction:transaction-api:jar:1.1:compile​ [INFO] | +- org.antlr:antlr-runtime:jar:3.5.2:compile​ [INFO] | +- org.apache.thrift:libfb303:jar:0.9.3:compile​ [INFO] | +- org.apache.thrift:libthrift:jar:0.9.3:compile​ [INFO] | | - org.apache.httpcomponents:httpcore:jar:4.4.1:compile​ [INFO] | +- co.cask.tephra:tephra-api:jar:0.6.0:compile​ [INFO] | +- co.cask.tephra:tephra-core:jar:0.6.0:compile​ [INFO] | | +- com.google.inject:guice:jar:3.0:compile​ [INFO] | | | +- javax.inject:javax.inject:jar:1:compile​ [INFO] | | | - aopalliance:aopalliance:jar:1.0:compile​ [INFO] | | +- com.google.inject.extensions:guice-assistedinject:jar:3.0:compile​ [INFO] | | +- it.unimi.dsi:fastutil:jar:6.5.6:compile​ [INFO] | | +- org.apache.twill:twill-common:jar:0.6.0-incubating:compile​ [INFO] | | +- org.apache.twill:twill-core:jar:0.6.0-incubating:compile​ [INFO] | | | - org.apache.twill:twill-api:jar:0.6.0-incubating:compile​ [INFO] | | +- org.apache.twill:twill-discovery-api:jar:0.6.0-incubating:compile​ [INFO] | | +- org.apache.twill:twill-discovery-core:jar:0.6.0-incubating:compile​ [INFO] | | +- org.apache.twill:twill-zookeeper:jar:0.6.0-incubating:compile​ [INFO] | | - io.dropwizard.metrics:metrics-core:jar:3.1.0:compile​ [INFO] | - co.cask.tephra:tephra-hbase-compat-1.0:jar:0.6.0:compile​ [INFO] - org.apache.hadoop:hadoop-common:jar:3.1.2:provided​ [INFO] +- org.apache.hadoop:hadoop-annotations:jar:3.1.2:compile​ [INFO] | - jdk.tools:jdk.tools:jar:1.8:system​ [INFO] +- org.apache.commons:commons-math3:jar:3.1.1:provided​ [INFO] +- org.apache.httpcomponents:httpclient:jar:4.5.2:compile​ [INFO] +- commons-codec:commons-codec:jar:1.11:compile​ [INFO] +- commons-io:commons-io:jar:2.5:compile​ [INFO] +- commons-net:commons-net:jar:3.6:provided​ [INFO] +- commons-collections:commons-collections:jar:3.2.2:compile​ [INFO] +- javax.servlet:javax.servlet-api:jar:3.1.0:provided​ [INFO] +- org.eclipse.jetty:jetty-server:jar:9.3.24.v20180605:provided​ [INFO] | +- org.eclipse.jetty:jetty-http:jar:9.3.24.v20180605:provided​ [INFO] | - org.eclipse.jetty:jetty-io:jar:9.3.24.v20180605:provided​ [INFO] +- org.eclipse.jetty:jetty-util:jar:9.3.24.v20180605:provided​ [INFO] +- org.eclipse.jetty:jetty-servlet:jar:9.3.24.v20180605:provided​ [INFO] | - org.eclipse.jetty:jetty-security:jar:9.3.24.v20180605:provided​ [INFO] +- org.eclipse.jetty:jetty-webapp:jar:9.3.24.v20180605:provided​ [INFO] | - org.eclipse.jetty:jetty-xml:jar:9.3.24.v20180605:provided​ [INFO] +- javax.servlet.jsp:jsp-api:jar:2.1:provided​ [INFO] +- com.sun.jersey:jersey-core:jar:1.19:runtime​ [INFO] | - javax.ws.rs:jsr311-api:jar:1.1.1:runtime​ [INFO] +- com.sun.jersey:jersey-servlet:jar:1.19:provided​ [INFO] +- com.sun.jersey:jersey-json:jar:1.19:runtime​ [INFO] | +- org.codehaus.jettison:jettison:jar:1.1:runtime​ [INFO] | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:runtime​ [INFO] | | - javax.xml.bind:jaxb-api:jar:2.2.2:compile​ [INFO] | | - javax.xml.stream:stax-api:jar:1.0-2:compile​ [INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.2:compile​ [INFO] | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.2:runtime​ [INFO] | - org.codehaus.jackson:jackson-xc:jar:1.9.2:runtime​ [INFO] +- com.sun.jersey:jersey-server:jar:1.19:runtime​ [INFO] +- commons-logging:commons-logging:jar:1.1.3:compile​ [INFO] +- log4j:log4j:jar:1.2.17:compile​ [INFO] +- commons-beanutils:commons-beanutils:jar:1.9.3:provided​ [INFO] +- org.apache.commons:commons-configuration2:jar:2.1.1:provided​ [INFO] +- org.apache.commons:commons-lang3:jar:3.4:compile​ [INFO] +- org.slf4j:slf4j-log4j12:jar:1.7.25:compile​ [INFO] +- org.apache.avro:avro:jar:1.7.7:compile​ [INFO] | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile​ [INFO] | - org.xerial.snappy:snappy-java:jar:1.0.5:compile​ [INFO] +- com.google.re2j:re2j:jar:1.1:provided​ [INFO] +- com.google.code.gson:gson:jar:2.2.4:compile​ [INFO] +- org.apache.hadoop:hadoop-auth:jar:3.1.2:compile​ [INFO] | +- com.nimbusds:nimbus-jose-jwt:jar:4.41.1:compile​ [INFO] | | - com.github.stephenc.jcip:jcip-annotations:jar:1.0-1:compile​ [INFO] | +- net.minidev:json-smart:jar:2.3:compile​ [INFO] | | - net.minidev:accessors-smart:jar:1.2:compile​ [INFO] | | - org.ow2.asm:asm:jar:5.0.4:compile​ [INFO] | - org.apache.curator:curator-framework:jar:2.13.0:compile​ [INFO] +- com.jcraft:jsch:jar:0.1.54:provided​ [INFO] +- org.apache.curator:curator-client:jar:2.13.0:compile​ [INFO] +- org.apache.curator:curator-recipes:jar:2.13.0:provided​ [INFO] +- com.google.code.findbugs:jsr305:jar:3.0.0:compile​ [INFO] +- org.apache.htrace:htrace-core4:jar:4.1.0-incubating:provided​ [INFO] +- org.apache.zookeeper:zookeeper:jar:3.4.13:compile​ [INFO] | +- jline:jline:jar:0.9.94:compile​ [INFO] | +- org.apache.yetus:audience-annotations:jar:0.5.0:compile​ [INFO] | - io.netty:netty:jar:3.10.6.Final:compile​ [INFO] +- org.apache.commons:commons-compress:jar:1.18:compile​ [INFO] +- org.apache.kerby:kerb-simplekdc:jar:1.0.1:compile​ [INFO] | +- org.apache.kerby:kerb-client:jar:1.0.1:compile​ [INFO] | | +- org.apache.kerby:kerby-config:jar:1.0.1:compile​ [INFO] | | +- org.apache.kerby:kerb-core:jar:1.0.1:compile​ [INFO] | | | - org.apache.kerby:kerby-pkix:jar:1.0.1:compile​ [INFO] | | | +- org.apache.kerby:kerby-asn1:jar:1.0.1:compile​ [INFO] | | | - org.apache.kerby:kerby-util:jar:1.0.1:compile​ [INFO] | | +- org.apache.kerby:kerb-common:jar:1.0.1:compile​ [INFO] | | | - org.apache.kerby:kerb-crypto:jar:1.0.1:compile​ [INFO] | | +- org.apache.kerby:kerb-util:jar:1.0.1:compile​ [INFO] | | - org.apache.kerby:token-provider:jar:1.0.1:compile​ [INFO] | - org.apache.kerby:kerb-admin:jar:1.0.1:compile​ [INFO] | +- org.apache.kerby:kerb-server:jar:1.0.1:compile​ [INFO] | | - org.apache.kerby:kerb-identity:jar:1.0.1:compile​ [INFO] | - org.apache.kerby:kerby-xdr:jar:1.0.1:compile​ [INFO] +- org.codehaus.woodstox:stax2-api:jar:3.1.4:provided​ [INFO] - com.fasterxml.woodstox:woodstox-core:jar:5.0.3:provided​ [INFO] ------------------------------------------------------------------------​ [INFO] Reactor Summary for egeria-connector-hms 1.5-SNAPSHOT:​ [INFO]​ [INFO] egeria-connector-hms ............................... SUCCESS [ 16.047 s]​ [INFO] adapter ............................................ SUCCESS [ 0.519 s]​ [INFO] ------------------------------------------------------------------------​ [INFO] BUILD SUCCESS​ [INFO] ------------------------------------------------------------------------​ [INFO] Total time: 17.281 s​ [INFO] Finished at: 2020-04-20T14:06:49-07:00​ [INFO] ------------------------------------------------------------------------

Sent from Outlookhttp://aka.ms/weboutlook

From: Nigel Jones notifications@github.com Sent: Monday, April 20, 2020 9:14 AM To: odpi/egeria egeria@noreply.github.com Cc: Shirong Liang shlia@microsoft.com; Author author@noreply.github.com Subject: Re: [odpi/egeria] add repository connector jar with hadoop as dependency, issue for Conflicting Logger modules. LoggerFactory is not a Logback LoggerContext but Logback is on the classpath (#2934)

I found I needed to build with an older jdk (I used Java 8) to avoid an issue with using 'tools.jar' (which is gone in later versions). Even then though I hit a compile error.

So perhaps let's just work off the errors.

I'm not experienced with hadoop libraries .. the class you mention is part of jetty, so I'm confused why excluding slf4j-log4j12 would cause that error

Can you get the output of 'mvn dependency:tree' of your module?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fodpi%2Fegeria%2Fissues%2F2934%23issuecomment-616656001&data=02%7C01%7Cshlia%40microsoft.com%7Cc947f2736b4d4cab311d08d7e545d795%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637229960447227588&sdata=HxUP1Oj24z5xDZTp%2FM2wIcr8erMHr58JihPzkJ5IjLg%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAENYGRC6WNM46YIIKJVHOX3RNRYEVANCNFSM4MKHL6DQ&data=02%7C01%7Cshlia%40microsoft.com%7Cc947f2736b4d4cab311d08d7e545d795%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637229960447227588&sdata=CGY6J3fPXWXrlGlmMJEf7gR5Hp0Ro%2BmOcXlCaa1127c%3D&reserved=0.

Shirong-Liang commented 4 years ago

Hi Nigel,

this it is the jetty server version conflict issue, the hadoop-common and egeria OMAG platform using this different version of jetty server.

Thanks!

planetf1 commented 4 years ago

Ok. glad you were able to make progress