search

openaire / iis

Information Inference Service of the OpenAIRE system
Apache License 2.0
19 stars 11 forks source link

SoftwareHeritage importer fails with NoSuchFieldError: INSTANCE #1122

Closed marekhorst closed 4 years ago

marekhorst commented 4 years ago

When trying to sync SH database SoftwareHeritageOriginsImporter surprisingly failed with:

aused by: java.lang.NoSuchFieldError: INSTANCE
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.<clinit>(SSLConnectionSocketFactory.java:151)
    at org.apache.http.impl.client.HttpClientBuilder.build(HttpClientBuilder.java:977)
    at eu.dnetlib.iis.wf.importer.software.origins.SoftwareHeritageOriginsImporter.buildHttpClient(SoftwareHeritageOriginsImporter.java:210)
    at eu.dnetlib.iis.wf.importer.software.origins.SoftwareHeritageOriginsImporter.run(SoftwareHeritageOriginsImporter.java:121)
    at eu.dnetlib.iis.common.java.ProcessWrapper.run(ProcessWrapper.java:49)

~After inspecting the changes in our code it looks like the problem was induced by introducing explicit httpclient dependency in main pom.xml file as a part of #1099 solution. Seems to be the clash between the different versions of http libraries.~ ~If this dependency could be dropped as a part of retracting the TARA caching solution in #1117 then we would probably automatically solve this issue. Otherwise we would need to dig deeper and align the http libraries versions.~

The httpclient explicit dependency was introduced as a part of #1070 major patent refactoring where HTTP communication with EPO endpoint was refactored and SH communication followed the same refactoring. Apparently the communication with SH endpoint is handled differently leading to the mentioned issue.

marekhorst commented 4 years ago

It seems both SSLConnectionSocketFactory and AllowAllHostnameVerifier (with INSTANCE reference) are part of the same lib: httpclient-4.5.12. However we have httpclient-4.2.5 and httpclient-3.1 coming from CDH5 parcels:

java.class.path=/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002\:/etc/hadoop/conf.cloudera.yarn2\:/run/cloudera-scm-agent/process/48221-yarn-NODEMANAGER\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-scala_2.10.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-thrift.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-azure-datalake-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-protobuf.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-pig-bundle.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-nfs.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-auth-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-encoding.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-test-hadoop2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-format-javadoc.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-common-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-nfs-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-pig.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-common-tests.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-scrooge_2.10.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-auth.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-annotations-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-tools.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-azure-datalake.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-hadoop-bundle.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-common.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-jackson.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-aws-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-format-sources.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-cascading.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-aws.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-hadoop.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-annotations.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-common.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-common-2.6.0-cdh5.16.2-tests.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-column.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-format.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-generator.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-avro.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/mockito-all-1.8.5.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-net-3.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/slf4j-log4j12.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/servlet-api-2.5.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jasper-compiler-5.5.23.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jaxb-api-2.2.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/avro.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-io-2.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jersey-server-1.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/xz-1.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jsp-api-2.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jetty-6.1.26.cloudera.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/httpclient-4.2.5.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jsch-0.1.42.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/logredactor-1.0.3.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/guava-11.0.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-digester-1.8.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jackson-xc-1.8.10.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/junit-4.11.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jasper-runtime-5.5.23.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/hamcrest-core-1.3.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jets3t-0.9.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/azure-data-lake-store-sdk-2.2.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/htrace-core4-4.0.1-incubating.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jersey-json-1.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/paranamer-2.3.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/log4j-1.2.17.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-el-1.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/java-xmlbuilder-0.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jetty-util-6.1.26.cloudera.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/curator-client-2.7.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/asm-3.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-collections-3.2.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-math3-3.1.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-logging-1.1.3.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/stax-api-1.0-2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/apacheds-i18n-2.0.0-M15.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-cli-1.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-compress-1.4.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jersey-core-1.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jackson-jaxrs-1.8.10.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/api-util-1.0.0-M20.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/httpcore-4.2.5.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/hue-plugins-3.9.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/netty-3.10.5.Final.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jettison-1.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/api-asn1-api-1.0.0-M20.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/gson-2.2.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-beanutils-1.9.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/curator-framework-2.7.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-httpclient-3.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-codec-1.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/zookeeper.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/slf4j-api-1.7.5.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/xmlenc-0.52.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-lang-2.6.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-configuration-1.6.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/snappy-java-1.0.4.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/curator-recipes-2.7.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/aws-java-sdk-bundle-1.11.134.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/activation-1.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jsr305-3.0.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/protobuf-java-2.5.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs-nfs.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs-2.6.0-cdh5.16.2-tests.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs-tests.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs-nfs-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/servlet-api-2.5.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-io-2.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jersey-server-1.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jsp-api-2.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/guava-11.0.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.10.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/htrace-core4-4.0.1-incubating.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/log4j-1.2.17.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-el-1.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/asm-3.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.10-cloudera.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-cli-1.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jersey-core-1.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/xercesImpl-2.9.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/netty-3.10.5.Final.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/xml-apis-1.3.04.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-codec-1.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/xmlenc-0.52.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-lang-2.6.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-tests.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-client.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-common.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-common-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-client-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-api-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-web-proxy-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-registry.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-nodemanager.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-api.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-tests-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-common-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-registry-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-common.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-web-proxy.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-nodemanager-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jline-2.11.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/servlet-api-2.5.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jaxb-api-2.2.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-io-2.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jersey-server-1.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/xz-1.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jetty-6.1.26.cloudera.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/guava-11.0.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jackson-xc-1.8.10.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jackson-core-asl-1.8.10.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/aopalliance-1.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/spark-1.6.0-cdh5.16.2-yarn-shuffle.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/leveldbjni-all-1.8.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jersey-json-1.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/log4j-1.2.17.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jetty-util-6.1.26.cloudera.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jersey-guice-1.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jersey-client-1.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/asm-3.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-collections-3.2.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jackson-mapper-asl-1.8.10-cloudera.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-logging-1.1.3.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/stax-api-1.0-2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-cli-1.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jersey-core-1.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jackson-jaxrs-1.8.10.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/guice-3.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jettison-1.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/javax.inject-1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-codec-1.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/zookeeper.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-lang-2.6.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/guice-servlet-3.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/spark-yarn-shuffle.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/activation-1.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jsr305-3.0.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/protobuf-java-2.5.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-extras-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-common.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/mockito-all-1.8.5.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-archives.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-gridmix-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/commons-net-3.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/servlet-api-2.5.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jasper-compiler-5.5.23.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jaxb-api-2.2.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-distcp-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/avro.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-datajoin-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/commons-io-2.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-azure.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jersey-server-1.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-app-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.16.2-tests.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/xz-1.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jsp-api-2.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-azure-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jetty-6.1.26.cloudera.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/httpclient-4.2.5.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-datajoin.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-archives-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jsch-0.1.42.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-auth-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/guava-11.0.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/commons-digester-1.8.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jackson-xc-1.8.10.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jackson-core-asl-1.8.10.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/okhttp-2.4.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-common-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-core-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/commons-beanutils-core-1.8.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/junit-4.11.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jackson-core-2.2.3.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jasper-runtime-5.5.23.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hamcrest-core-1.3.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jets3t-0.9.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/htrace-core4-4.0.1-incubating.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-tests.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jersey-json-1.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/paranamer-2.3.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/log4j-1.2.17.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jackson-databind-2.2.3-cloudera.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-nativetask.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-archive-logs-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/commons-el-1.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/java-xmlbuilder-0.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-rumen-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-ant.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-auth.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jetty-util-6.1.26.cloudera.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/curator-client-2.7.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/asm-3.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/commons-collections-3.2.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jackson-mapper-asl-1.8.10-cloudera.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-nativetask-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/apacheds-kerberos-codec-2.0.0-M15.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/commons-math3-3.1.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-openstack.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/commons-logging-1.1.3.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/stax-api-1.0-2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/apacheds-i18n-2.0.0-M15.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/commons-cli-1.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/okio-1.4.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-streaming.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/commons-compress-1.4.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-extras.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jersey-core-1.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jackson-jaxrs-1.8.10.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/api-util-1.0.0-M20.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/httpcore-4.2.5.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-openstack-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-sls.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-ant-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-app.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jettison-1.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jackson-annotations-2.2.3.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/api-asn1-api-1.0.0-M20.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/gson-2.2.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/commons-beanutils-1.9.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/curator-framework-2.7.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/commons-httpclient-3.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/commons-codec-1.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/metrics-core-3.0.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-streaming-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/zookeeper.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jaxb-impl-2.2.3-1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/xmlenc-0.52.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/commons-lang-2.6.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/microsoft-windowsazure-storage-sdk-0.6.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/commons-configuration-1.6.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-archive-logs.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/snappy-java-1.0.4.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-distcp.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/curator-recipes-2.7.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-rumen.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/activation-1.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/jsr305-3.0.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-gridmix.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/hadoop-sls-2.6.0-cdh5.16.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/protobuf-java-2.5.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/avro.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/commons-io-2.4.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/xz-1.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/jackson-core-asl-1.8.10.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/junit-4.11.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/hamcrest-core-1.3.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/leveldbjni-all-1.8.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/paranamer-2.3.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/asm-3.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.8.10-cloudera.2.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/guice-3.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/netty-3.10.5.Final.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/javax.inject-1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar\:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar\:\:job.jar/job.jar\:job.jar/classes/\:job.jar/lib/*\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/super-csv-2.4.0.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cxf-rt-frontend-simple-3.1.5.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/avro-ipc-1.7.6-cdh5.16.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/dnet-pace-core-3.0.15.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/jackson-mapper-asl-1.8.10-cloudera.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/xml-resolver-1.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/noggit-0.8.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/slf4j-api-1.7.22.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/iis-3rdparty-avro-json-1.2.0-SNAPSHOT.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/spring-core-4.2.3.RELEASE.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/iis-wf-referenceextraction-1.2.0-SNAPSHOT.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/commons-logging-1.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/oozie-sharelib-oozie.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/spring-test-4.2.5.RELEASE.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/datafu-1.2.0.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/spring-context-support-4.2.5.RELEASE.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/xmlpull-1.1.3.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/jackson-core-asl-1.8.10.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/dnet-index-solr-common-2.3.4.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/apacheds-i18n-2.0.0-M15.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/velocity-1.7.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/istack-commons-runtime-2.16.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/quartz-2.2.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/curator-framework-2.7.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/aws-java-sdk-bundle-1.11.134.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/guava-12.0.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/spring-context-4.2.5.RELEASE.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/iis-schemas-1.2.0-SNAPSHOT.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/snappy-java-1.1.7.3.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/antlr-2.7.7.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/commons-math3-3.4.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/spring-tx-4.2.5.RELEASE.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/oro-2.0.8.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/xz-1.0.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/httpclient-4.5.12.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cxf-rt-databinding-jaxb-3.1.5.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/log4j-1.2.17.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/dnet-openaireplus-mapping-utils-6.3.43.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/apacheds-kerberos-codec-2.0.0-M15.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cnr-spring-utils-1.0.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/iis-wf-import-1.2.0-SNAPSHOT.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cnr-service-common-2.1.7.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/zookeeper-3.5.1-alpha.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/dhp-schemas-1.2.3.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cxf-rt-bindings-soap-3.1.5.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/stringtemplate-3.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/avro-mapred-1.7.6-cdh5.16.2-hadoop2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/netty-3.9.9.Final.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cxf-rt-ws-addr-3.1.5.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/woodstox-core-asl-4.4.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/jaxen-1.1.6.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/oozie-client-4.1.0-cdh5.16.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/xmlschema-core-2.2.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/spring-expression-4.2.5.RELEASE.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/xpp3_min-1.1.4c.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/FastInfoset-1.2.12.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/commons-cli-1.3.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/xstream-1.4.9.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/commons-collections-3.2.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cnr-service-utils-1.0.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/hawtbuf-1.11.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/vtd-xml-2.13.4.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/slf4j-log4j12-1.7.22.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/reflections-0.9.10.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/dnet-hadoop-commons-2.0.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/protobuf-java-2.5.0.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/oozie-hadoop-utils-2.6.0-cdh5.16.2.oozie-4.1.0-cdh5.16.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/json-simple-1.1.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/secondstring-1.0.0.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/avro-ipc-1.7.6-cdh5.16.2-tests.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/commons-beanutils-1.9.3.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/iis-3rdparty-madis-1.2.0-SNAPSHOT.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/dnet-openaire-data-protos-3.9.8.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/commons-codec-1.11.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/solr-solrj-7.5.0.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/commons-lang-2.6.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/geronimo-j2ee-management_1.1_spec-1.0.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/saxon-9.1.0.8.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/paranamer-2.8.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/jcommander-1.60.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/javax.servlet-api-3.1.0.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/jdom-1.1.3.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/dom4j-1.6.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/commons-lang3-3.5.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/saxon-dom-9.1.0.8.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/iis-common-1.2.0-SNAPSHOT.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/jetty-util-6.1.26.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/wsdl4j-1.6.3.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/runcc-0.7.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/hadoop-aws-2.6.0-cdh5.16.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/httpmime-4.5.3.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/ehcache-2.8.0.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cxf-rt-transports-http-3.1.5.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/dnet-objectstore-rmi-2.0.10.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/avro-1.7.6-cdh5.16.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/json-simple-1.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/joda-time-1.6.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/activemq-client-5.13.3.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/commons-io-2.5.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/iis-3rdparty-avrojsoncoders-1.2.0-SNAPSHOT.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cxf-rt-frontend-jaxws-3.1.5.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/jaxb-api-2.2.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/oozie-hadoop-utils.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/asm-5.0.4.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/hadoop-auth-2.6.0-cdh5.16.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cxf-rt-ws-policy-3.1.5.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/neethi-3.0.3.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/iis-build-assembly-resources-1.2.0-SNAPSHOT.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/jaxb-core-2.2.7.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/api-asn1-api-1.0.0-M20.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/curator-client-2.7.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/geronimo-jms_1.1_spec-1.1.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cnr-rmi-api-2.6.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/jetty-6.1.26.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/protobuf-java-format-1.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/xml-apis-1.4.01.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cxf-core-3.1.5.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/oozie-sharelib-oozie-4.1.0-cdh5.16.2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/httpcore-4.4.13.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/xercesImpl-2.11.0.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/jaxb-impl-2.2.7.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/spring-beans-4.2.5.RELEASE.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/iis-wf-transformers-1.2.0-SNAPSHOT.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/commons-compress-1.4.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/api-util-1.0.0-M20.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cnr-misc-utils-1.0.7.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cxf-rt-bindings-xml-3.1.5.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/jsr173_api-1.0.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/c3p0-0.9.1.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/commons-collections4-4.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/activation-1.1.1.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/stax-api-1.0-2.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/spring-aop-4.2.5.RELEASE.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/stax2-api-3.1.4.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/javacc-5.0.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cnr-resultset-service-2.0.3.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/gson-2.8.0.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/javassist-3.21.0-GA.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cnr-resultset-client-2.1.6.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/cxf-rt-wsdl-3.1.5.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/jparsec-2.0.jar\:/data/1/yarn/nm/usercache/marek.horst/appcache/application_1595483222400_5239/container_e70_1595483222400_5239_01_000002/aopalliance-1.0.jar

which might cause problems.

The classes are being loaded from the classpath in the order they are defined: httpclient-4.2.5, httpclient-3.1, httpclient-4.5.12. The class SSLConnectionSocketFactory is defined only in the most recent lib while AllowAllHostnameVerifier is also available in httpclient-4.2.5 and has no INSTANCE field defined.

Not sure why the problem did not show up for the patent mining which is defined in the same submodule (iis-wf-referenceextraction) it is possible the classpath order was different.

EDIT: it might be caused by the spark nature of the patents metadata retriever where the classpath order is different:

http://iis-cdh5-test-m3.ocean.icm.edu.pl:18089/history/application_1595483222400_4367/1/environment/

where the first classpath entry brings httpclient-4.5.12.

marekhorst commented 4 years ago

When inspecting mapreduce.job.classpath.files property contents on MapReduce configuration:

http://iis-cdh5-test-m3.ocean.icm.edu.pl:19888/jobhistory/conf/job_1595483222400_5239

it clearly shows only one httpclient-4.5.12 entry in the classpath while the classpath of the job is "polluted" with various httpclient versions coming from parcels: http://iis-cdh5-test-m3.ocean.icm.edu.pl:19888/jobhistory/logs/eos-m2-sn02.ocean.icm.edu.pl:8041/container_e70_1595483222400_5239_01_000002/attempt_1595483222400_5239_m_000000_0/marek.horst/stdout/?start=0

One thing we could do is to make sure the libs provided within an oozie package has precedence over the parcel libs which is the case for spark jobs and which seems to be quite expected and reasonable.

marekhorst commented 4 years ago

If we fail to solve this issue we might need to revert this fix:

https://github.com/openaire/iis/commit/99eeaabaf5f6c6fa28e4e9a5c39af1bdc8e9b348#diff-9714a9954d9d4e1c0262034e2219e6f6

The reason it was introduced was the issue with certificates handling. As a result of this issue we were unable to communicate with https endpoint so we had to stick to http.

More details in: https://issue.openaire.research-infrastructures.eu/issues/5673#note-9

marekhorst commented 4 years ago

Found this precious thread:

https://community.cloudera.com/t5/Support-Questions/Oozie-workflow-trying-to-give-my-jar-precedence-in-shell/td-p/213722

setting this property in the workflow.xml definition:

<property>
    <name>oozie.launcher.mapreduce.user.classpath.first</name>
    <value>true</value>
</property>

works as expected and solves this issue.