search

apache / hudi

Upserts, Deletes And Incremental Processing on Big Data.
https://hudi.apache.org/
Apache License 2.0
5.31k stars 2.41k forks source link

[SUPPORT] PySpark Quickstart Guide fails on Insert Data with `java.lang.UnsatisfiedLinkError` for Hudi 0.8.0 or 0.7.0 & PySpark 3.1.2 #3492

Closed jslorrma closed 3 years ago

jslorrma commented 3 years ago

Tips before filing an issue

Describe the problem you faced

Hudi fails during the PySpark Quickstart Guide in step 2 when inserting data. I have tried version 0.7.0 and 0.8.0 by loading hudi-spark-bundle_2.12-0.7.0.jar or hudi-spark-bundle_2.12-0.8.0.jar together with spark-avro_2.12-3.1.2.jar during the SparkSession intialisation. My PySpark version is 3.1.2 (Scala 2.12).

I can see, that some data is written to parquet-files as well as the expected partitioning folder folder structure is there.

ls -la -R /tmp/hudi_trips_cow shows: ```bash ls -la -R /tmp/hudi_trips_cow /tmp/hudi_trips_cow: total 0 drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 . drwxrwxrwt 1 root root 4096 Aug 17 18:11 .. drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 americas drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 asia drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 .hoodie /tmp/hudi_trips_cow/americas: total 0 drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 . drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 .. drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 brazil drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 united_states /tmp/hudi_trips_cow/americas/brazil: total 0 drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 . drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 .. drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 sao_paulo /tmp/hudi_trips_cow/americas/brazil/sao_paulo: total 452 drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 . drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 .. -rw-rw---- 1 jslorrma jslorrma 437416 Aug 17 18:11 37a6381a-b93b-4123-8389-6a9089865762-0_0-22-28_20210817181059.parquet -rw-rw---- 1 jslorrma jslorrma 3428 Aug 17 18:11 .37a6381a-b93b-4123-8389-6a9089865762-0_0-22-28_20210817181059.parquet.crc -rw-rw---- 1 jslorrma jslorrma 94 Aug 17 18:11 .hoodie_partition_metadata -rw-rw---- 1 jslorrma jslorrma 12 Aug 17 18:11 ..hoodie_partition_metadata.crc .. .. .. /tmp/hudi_trips_cow/.hoodie: total 4 drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 . drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 .. -rw-rw---- 1 jslorrma jslorrma 0 Aug 17 18:11 20210817181059.commit.requested -rw-rw---- 1 jslorrma jslorrma 8 Aug 17 18:11 .20210817181059.commit.requested.crc -rw-rw---- 1 jslorrma jslorrma 2594 Aug 17 18:11 20210817181059.inflight -rw-rw---- 1 jslorrma jslorrma 32 Aug 17 18:11 .20210817181059.inflight.crc drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:10 archived drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:10 .aux -rw-rw---- 1 jslorrma jslorrma 270 Aug 17 18:10 hoodie.properties -rw-rw---- 1 jslorrma jslorrma 12 Aug 17 18:10 .hoodie.properties.crc drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 .temp /tmp/hudi_trips_cow/.hoodie/archived: total 0 drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:10 . drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 .. /tmp/hudi_trips_cow/.hoodie/.aux: total 0 drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:10 . drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 .. drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:10 .bootstrap /tmp/hudi_trips_cow/.hoodie/.aux/.bootstrap: total 0 drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:10 . drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:10 .. drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:10 .fileids drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:10 .partitions /tmp/hudi_trips_cow/.hoodie/.aux/.bootstrap/.fileids: total 0 drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:10 . drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:10 .. /tmp/hudi_trips_cow/.hoodie/.aux/.bootstrap/.partitions: total 0 drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:10 . drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:10 .. /tmp/hudi_trips_cow/.hoodie/.temp: total 0 drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 . drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 .. drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 20210817181059 /tmp/hudi_trips_cow/.hoodie/.temp/20210817181059: total 0 drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 . drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 .. drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 americas drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 asia /tmp/hudi_trips_cow/.hoodie/.temp/20210817181059/americas: total 0 drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 . drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 .. drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 brazil drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 united_states /tmp/hudi_trips_cow/.hoodie/.temp/20210817181059/americas/brazil: total 0 drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 . drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 .. drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 sao_paulo /tmp/hudi_trips_cow/.hoodie/.temp/20210817181059/americas/brazil/sao_paulo: total 0 drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 . drwxrwx--x 1 jslorrma jslorrma 4096 Aug 17 18:11 .. -rw-rw---- 1 jslorrma jslorrma 0 Aug 17 18:11 37a6381a-b93b-4123-8389-6a9089865762-0_0-22-28_20210817181059.parquet.marker.CREATE -rw-rw---- 1 jslorrma jslorrma 8 Aug 17 18:11 .37a6381a-b93b-4123-8389-6a9089865762-0_0-22-28_20210817181059.parquet.marker.CREATE.crc .. .. .. ```

For the example below the job fails in Stage 29 Obtaining marker files for all created, merged paths collect at HoodieSparkEngineContext.java:78 with the following error message:

Job aborted due to stage failure: Task 1 in stage 29.0 failed 1 times, most recent failure: Lost task 1.0 in stage 29.0 (TID 35) (10.142.245.45 executor driver): java.lang.UnsatisfiedLinkError: 'org.apache.hadoop.io.nativeio.NativeIO$POSIX$Stat org.apache.hadoop.io.nativeio.NativeIO$POSIX.stat(java.lang.String)'
    at org.apache.hadoop.io.nativeio.NativeIO$POSIX.stat(Native Method)
    at org.apache.hadoop.io.nativeio.NativeIO$POSIX.getStat(NativeIO.java:456)
    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfoByNativeIO(RawLocalFileSystem.java:821)
    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:735)
    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.getPermission(RawLocalFileSystem.java:703)
    at org.apache.hadoop.fs.LocatedFileStatus.<init>(LocatedFileStatus.java:52)
    at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:2091)
    at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:2071)
    at org.apache.hadoop.fs.FileSystem$5.hasNext(FileSystem.java:2190)
    at org.apache.hudi.table.MarkerFiles.lambda$createdAndMergedDataPaths$69cdea3b$1(MarkerFiles.java:144)
    at org.apache.hudi.client.common.HoodieSparkEngineContext.lambda$flatMap$7d470b86$1(HoodieSparkEngineContext.java:78)
    at org.apache.spark.api.java.JavaRDDLike.$anonfun$flatMap$1(JavaRDDLike.scala:125)
    at scala.collection.Iterator$$anon$11.nextCur(Iterator.scala:484)
    at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:490)
    at scala.collection.Iterator.foreach(Iterator.scala:941)
    at scala.collection.Iterator.foreach$(Iterator.scala:941)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
    at scala.collection.generic.Growable.$plus$plus$eq(Growable.scala:62)
    at scala.collection.generic.Growable.$plus$plus$eq$(Growable.scala:53)
    at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:105)
    at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:49)
    at scala.collection.TraversableOnce.to(TraversableOnce.scala:315)
    at scala.collection.TraversableOnce.to$(TraversableOnce.scala:313)
    at scala.collection.AbstractIterator.to(Iterator.scala:1429)
    at scala.collection.TraversableOnce.toBuffer(TraversableOnce.scala:307)
    at scala.collection.TraversableOnce.toBuffer$(TraversableOnce.scala:307)
    at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1429)
    at scala.collection.TraversableOnce.toArray(TraversableOnce.scala:294)
    at scala.collection.TraversableOnce.toArray$(TraversableOnce.scala:288)
    at scala.collection.AbstractIterator.toArray(Iterator.scala:1429)
    at org.apache.spark.rdd.RDD.$anonfun$collect$2(RDD.scala:1030)
    at org.apache.spark.SparkContext.$anonfun$runJob$5(SparkContext.scala:2236)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:131)
    at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)

To Reproduce

Steps to reproduce the behavior:

  1. Configure and initialize SparkSession by 
    
# imports
import sys
import os
import pyspark

set env vars

os.environ["PYSPARK_PYTHON"] = sys.executable os.environ["PYSPARK_DRIVER_PYTHON"] = sys.executable

conf = ( pyspark.SparkConf() .set("spark.app.name", "hudi test" ) .set("spark.master", "local[*]") .set("spark.submit.deployMode", "client")

Path to the archived Python environment

# Pyspark configuration of application master and worker nodes
.set("spark.pyspark.python", sys.executable)
.set("spark.pyspark.driver.python", sys.executable)
#.set("spark.jars", _jars)
.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer")
# memory and executor configuration
.set("spark.dynamicAllocation.enabled", "true")
.set("spark.dynamicAllocation.minExecutors", "1")
.set("spark.dynamicAllocation.maxExecutors", "60")    
#.set("spark.executor.instances","6")
.set("spark.executor.memory","6g")
.set("spark.executor.memoryOverhead", "3g")
.set("spark.executor.cores", "5")
.set("spark.driver.memory", "20g")
.set("spark.driver.memoryOverhead", "8g")
.set("spark.driver.cores", "4")
# optimization
.set("spark.sql.execution.arrow.pyspark.enabled", "true")
.set("spark.sql.execution.arrow.pyspark.fallback.enabled", "false")

)

from pyspark.sql import SparkSession spark = ( SparkSession.builder .config(conf=conf) .enableHiveSupport() .getOrCreate() )

show the Info about the spark session

spark.sparkContext

The output is:
```python
Out[1]: 
    SparkContext
    Spark UI
    Version
        v3.1.2
    Master
        local[*]
    AppName
        hudi test
  1. Insert data 
    # pyspark sparkContext
sc = spark.sparkContext
# table name and table path
tableName = "hudi_trips_cow"
basePath = "file:///tmp/hudi_trips_cow"
# generate sample data
dataGen = sc._jvm.org.apache.hudi.QuickstartUtils.DataGenerator()
inserts = sc._jvm.org.apache.hudi.QuickstartUtils.convertToStringList(
dataGen.generateInserts(10)
)
df = spark.read.json(
spark.sparkContext.parallelize(inserts, 2)
)
# configure hudi options
hudi_options = {    
'hoodie.table.name': tableName,    
'hoodie.datasource.write.recordkey.field': 'uuid',    
'hoodie.datasource.write.partitionpath.field': 'partitionpath',    
'hoodie.datasource.write.table.name': tableName,    
'hoodie.datasource.write.operation': 'upsert',    
'hoodie.datasource.write.precombine.field': 'ts',    
'hoodie.upsert.shuffle.parallelism': 2,   
'hoodie.insert.shuffle.parallelism': 2
}
# write hudi table
df.write.format("hudi")\
.options(**hudi_options)\
.mode("overwrite")\
.save(basePath)

Expected behavior

No error message during data insertion.

Environment Description

Additional context

As I'm always a bit lost with the spark / java error-traces and I don't want to overload this first posting, I just add some additional context here and not more error / stack-traces, than what I've already posted above. If anything else is needed please ask!

Classpath entries (including hudi-spark-bundle_2.12-0.8.0.jar and spark-avro_2.12-3.1.2.jar ): ``` /etc/hadoop/conf/ /etc/hive/conf/ /home/jslorrma/.spark3/conf/ /home/jslorrma/.spark3/jars/HikariCP-2.5.1.jar /home/jslorrma/.spark3/jars/JLargeArrays-1.5.jar /home/jslorrma/.spark3/jars/JTransforms-3.1.jar /home/jslorrma/.spark3/jars/RoaringBitmap-0.9.0.jar /home/jslorrma/.spark3/jars/ST4-4.0.4.jar /home/jslorrma/.spark3/jars/accessors-smart-1.2.jar /home/jslorrma/.spark3/jars/activation-1.1.1.jar /home/jslorrma/.spark3/jars/aircompressor-0.10.jar /home/jslorrma/.spark3/jars/algebra_2.12-2.0.0-M2.jar /home/jslorrma/.spark3/jars/antlr-runtime-3.5.2.jar /home/jslorrma/.spark3/jars/antlr4-runtime-4.8-1.jar /home/jslorrma/.spark3/jars/aopalliance-1.0.jar /home/jslorrma/.spark3/jars/aopalliance-repackaged-2.6.1.jar /home/jslorrma/.spark3/jars/arpack_combined_all-0.1.jar /home/jslorrma/.spark3/jars/arrow-format-2.0.0.jar /home/jslorrma/.spark3/jars/arrow-memory-core-2.0.0.jar /home/jslorrma/.spark3/jars/arrow-memory-netty-2.0.0.jar /home/jslorrma/.spark3/jars/arrow-vector-2.0.0.jar /home/jslorrma/.spark3/jars/audience-annotations-0.5.0.jar /home/jslorrma/.spark3/jars/automaton-1.11-8.jar /home/jslorrma/.spark3/jars/avro-1.8.2.jar /home/jslorrma/.spark3/jars/avro-ipc-1.8.2.jar /home/jslorrma/.spark3/jars/avro-mapred-1.8.2-hadoop2.jar /home/jslorrma/.spark3/jars/bonecp-0.8.0.RELEASE.jar /home/jslorrma/.spark3/jars/breeze-macros_2.12-1.0.jar /home/jslorrma/.spark3/jars/breeze_2.12-1.0.jar /home/jslorrma/.spark3/jars/cats-kernel_2.12-2.0.0-M4.jar /home/jslorrma/.spark3/jars/chill-java-0.9.5.jar /home/jslorrma/.spark3/jars/chill_2.12-0.9.5.jar /home/jslorrma/.spark3/jars/commons-beanutils-1.9.4.jar /home/jslorrma/.spark3/jars/commons-cli-1.2.jar /home/jslorrma/.spark3/jars/commons-codec-1.10.jar /home/jslorrma/.spark3/jars/commons-collections-3.2.2.jar /home/jslorrma/.spark3/jars/commons-compiler-3.0.16.jar /home/jslorrma/.spark3/jars/commons-compress-1.20.jar /home/jslorrma/.spark3/jars/commons-configuration2-2.1.1.jar /home/jslorrma/.spark3/jars/commons-crypto-1.1.0.jar /home/jslorrma/.spark3/jars/commons-daemon-1.0.13.jar /home/jslorrma/.spark3/jars/commons-dbcp-1.4.jar /home/jslorrma/.spark3/jars/commons-httpclient-3.1.jar /home/jslorrma/.spark3/jars/commons-io-2.5.jar /home/jslorrma/.spark3/jars/commons-lang-2.6.jar /home/jslorrma/.spark3/jars/commons-lang3-3.10.jar /home/jslorrma/.spark3/jars/commons-logging-1.1.3.jar /home/jslorrma/.spark3/jars/commons-math3-3.4.1.jar /home/jslorrma/.spark3/jars/commons-net-3.1.jar /home/jslorrma/.spark3/jars/commons-pool-1.5.4.jar /home/jslorrma/.spark3/jars/commons-text-1.6.jar /home/jslorrma/.spark3/jars/compress-lzf-1.0.3.jar /home/jslorrma/.spark3/jars/core-1.1.2.jar /home/jslorrma/.spark3/jars/curator-client-2.13.0.jar /home/jslorrma/.spark3/jars/curator-framework-2.13.0.jar /home/jslorrma/.spark3/jars/curator-recipes-2.13.0.jar /home/jslorrma/.spark3/jars/datanucleus-api-jdo-4.2.4.jar /home/jslorrma/.spark3/jars/datanucleus-core-4.1.17.jar /home/jslorrma/.spark3/jars/datanucleus-rdbms-4.1.19.jar /home/jslorrma/.spark3/jars/derby-10.12.1.1.jar /home/jslorrma/.spark3/jars/dnsjava-2.1.7.jar /home/jslorrma/.spark3/jars/dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar /home/jslorrma/.spark3/jars/ehcache-3.3.1.jar /home/jslorrma/.spark3/jars/flatbuffers-java-1.9.0.jar /home/jslorrma/.spark3/jars/generex-1.0.2.jar /home/jslorrma/.spark3/jars/geronimo-jcache_1.0_spec-1.0-alpha-1.jar /home/jslorrma/.spark3/jars/gson-2.2.4.jar /home/jslorrma/.spark3/jars/guava-14.0.1.jar /home/jslorrma/.spark3/jars/guice-4.0.jar /home/jslorrma/.spark3/jars/guice-servlet-4.0.jar /home/jslorrma/.spark3/jars/hadoop-annotations-3.2.0.jar /home/jslorrma/.spark3/jars/hadoop-auth-3.2.0.jar /home/jslorrma/.spark3/jars/hadoop-client-3.2.0.jar /home/jslorrma/.spark3/jars/hadoop-common-3.2.0.jar /home/jslorrma/.spark3/jars/hadoop-hdfs-client-3.2.0.jar /home/jslorrma/.spark3/jars/hadoop-mapreduce-client-common-3.2.0.jar /home/jslorrma/.spark3/jars/hadoop-mapreduce-client-core-3.2.0.jar /home/jslorrma/.spark3/jars/hadoop-mapreduce-client-jobclient-3.2.0.jar /home/jslorrma/.spark3/jars/hadoop-yarn-api-3.2.0.jar /home/jslorrma/.spark3/jars/hadoop-yarn-client-3.2.0.jar /home/jslorrma/.spark3/jars/hadoop-yarn-common-3.2.0.jar /home/jslorrma/.spark3/jars/hadoop-yarn-registry-3.2.0.jar /home/jslorrma/.spark3/jars/hadoop-yarn-server-common-3.2.0.jar /home/jslorrma/.spark3/jars/hadoop-yarn-server-web-proxy-3.2.0.jar /home/jslorrma/.spark3/jars/hive-beeline-2.3.7.jar /home/jslorrma/.spark3/jars/hive-cli-2.3.7.jar /home/jslorrma/.spark3/jars/hive-common-2.3.7.jar /home/jslorrma/.spark3/jars/hive-exec-2.3.7-core.jar /home/jslorrma/.spark3/jars/hive-jdbc-2.3.7.jar /home/jslorrma/.spark3/jars/hive-llap-common-2.3.7.jar /home/jslorrma/.spark3/jars/hive-metastore-2.3.7.jar /home/jslorrma/.spark3/jars/hive-serde-2.3.7.jar /home/jslorrma/.spark3/jars/hive-service-rpc-3.1.2.jar /home/jslorrma/.spark3/jars/hive-shims-0.23-2.3.7.jar /home/jslorrma/.spark3/jars/hive-shims-2.3.7.jar /home/jslorrma/.spark3/jars/hive-shims-common-2.3.7.jar /home/jslorrma/.spark3/jars/hive-shims-scheduler-2.3.7.jar /home/jslorrma/.spark3/jars/hive-storage-api-2.7.2.jar /home/jslorrma/.spark3/jars/hive-vector-code-gen-2.3.7.jar /home/jslorrma/.spark3/jars/hk2-api-2.6.1.jar /home/jslorrma/.spark3/jars/hk2-locator-2.6.1.jar /home/jslorrma/.spark3/jars/hk2-utils-2.6.1.jar /home/jslorrma/.spark3/jars/htrace-core4-4.1.0-incubating.jar /home/jslorrma/.spark3/jars/httpclient-4.5.6.jar /home/jslorrma/.spark3/jars/httpcore-4.4.12.jar /home/jslorrma/.spark3/jars/hudi-spark-bundle_2.12-0.8.0.jar /home/jslorrma/.spark3/jars/istack-commons-runtime-3.0.8.jar /home/jslorrma/.spark3/jars/ivy-2.4.0.jar /home/jslorrma/.spark3/jars/jackson-annotations-2.10.0.jar /home/jslorrma/.spark3/jars/jackson-core-2.10.0.jar /home/jslorrma/.spark3/jars/jackson-core-asl-1.9.13.jar /home/jslorrma/.spark3/jars/jackson-databind-2.10.0.jar /home/jslorrma/.spark3/jars/jackson-dataformat-yaml-2.10.0.jar /home/jslorrma/.spark3/jars/jackson-datatype-jsr310-2.11.2.jar /home/jslorrma/.spark3/jars/jackson-jaxrs-base-2.9.5.jar /home/jslorrma/.spark3/jars/jackson-jaxrs-json-provider-2.9.5.jar /home/jslorrma/.spark3/jars/jackson-mapper-asl-1.9.13.jar /home/jslorrma/.spark3/jars/jackson-module-jaxb-annotations-2.10.0.jar /home/jslorrma/.spark3/jars/jackson-module-paranamer-2.10.0.jar /home/jslorrma/.spark3/jars/jackson-module-scala_2.12-2.10.0.jar /home/jslorrma/.spark3/jars/jakarta.activation-api-1.2.1.jar /home/jslorrma/.spark3/jars/jakarta.annotation-api-1.3.5.jar /home/jslorrma/.spark3/jars/jakarta.inject-2.6.1.jar /home/jslorrma/.spark3/jars/jakarta.servlet-api-4.0.3.jar /home/jslorrma/.spark3/jars/jakarta.validation-api-2.0.2.jar /home/jslorrma/.spark3/jars/jakarta.ws.rs-api-2.1.6.jar /home/jslorrma/.spark3/jars/jakarta.xml.bind-api-2.3.2.jar /home/jslorrma/.spark3/jars/janino-3.0.16.jar /home/jslorrma/.spark3/jars/javassist-3.25.0-GA.jar /home/jslorrma/.spark3/jars/javax.inject-1.jar /home/jslorrma/.spark3/jars/javax.jdo-3.2.0-m3.jar /home/jslorrma/.spark3/jars/javolution-5.5.1.jar /home/jslorrma/.spark3/jars/jaxb-api-2.2.11.jar /home/jslorrma/.spark3/jars/jaxb-runtime-2.3.2.jar /home/jslorrma/.spark3/jars/jcip-annotations-1.0-1.jar /home/jslorrma/.spark3/jars/jcl-over-slf4j-1.7.30.jar /home/jslorrma/.spark3/jars/jdo-api-3.0.1.jar /home/jslorrma/.spark3/jars/jersey-client-2.30.jar /home/jslorrma/.spark3/jars/jersey-common-2.30.jar /home/jslorrma/.spark3/jars/jersey-container-servlet-2.30.jar /home/jslorrma/.spark3/jars/jersey-container-servlet-core-2.30.jar /home/jslorrma/.spark3/jars/jersey-hk2-2.30.jar /home/jslorrma/.spark3/jars/jersey-media-jaxb-2.30.jar /home/jslorrma/.spark3/jars/jersey-server-2.30.jar /home/jslorrma/.spark3/jars/jline-2.14.6.jar /home/jslorrma/.spark3/jars/joda-time-2.10.5.jar /home/jslorrma/.spark3/jars/jodd-core-3.5.2.jar /home/jslorrma/.spark3/jars/jpam-1.1.jar /home/jslorrma/.spark3/jars/json-1.8.jar /home/jslorrma/.spark3/jars/json-smart-2.3.jar /home/jslorrma/.spark3/jars/json4s-ast_2.12-3.7.0-M5.jar /home/jslorrma/.spark3/jars/json4s-core_2.12-3.7.0-M5.jar /home/jslorrma/.spark3/jars/json4s-jackson_2.12-3.7.0-M5.jar /home/jslorrma/.spark3/jars/json4s-scalap_2.12-3.7.0-M5.jar /home/jslorrma/.spark3/jars/jsp-api-2.1.jar /home/jslorrma/.spark3/jars/jsr305-3.0.0.jar /home/jslorrma/.spark3/jars/jta-1.1.jar /home/jslorrma/.spark3/jars/jul-to-slf4j-1.7.30.jar /home/jslorrma/.spark3/jars/kerb-admin-1.0.1.jar /home/jslorrma/.spark3/jars/kerb-client-1.0.1.jar /home/jslorrma/.spark3/jars/kerb-common-1.0.1.jar /home/jslorrma/.spark3/jars/kerb-core-1.0.1.jar /home/jslorrma/.spark3/jars/kerb-crypto-1.0.1.jar /home/jslorrma/.spark3/jars/kerb-identity-1.0.1.jar /home/jslorrma/.spark3/jars/kerb-server-1.0.1.jar /home/jslorrma/.spark3/jars/kerb-simplekdc-1.0.1.jar /home/jslorrma/.spark3/jars/kerb-util-1.0.1.jar /home/jslorrma/.spark3/jars/kerby-asn1-1.0.1.jar /home/jslorrma/.spark3/jars/kerby-config-1.0.1.jar /home/jslorrma/.spark3/jars/kerby-pkix-1.0.1.jar /home/jslorrma/.spark3/jars/kerby-util-1.0.1.jar /home/jslorrma/.spark3/jars/kerby-xdr-1.0.1.jar /home/jslorrma/.spark3/jars/kryo-shaded-4.0.2.jar /home/jslorrma/.spark3/jars/kubernetes-client-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-admissionregistration-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-apiextensions-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-apps-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-autoscaling-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-batch-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-certificates-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-common-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-coordination-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-core-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-discovery-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-events-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-extensions-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-metrics-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-networking-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-policy-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-rbac-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-scheduling-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-settings-4.12.0.jar /home/jslorrma/.spark3/jars/kubernetes-model-storageclass-4.12.0.jar /home/jslorrma/.spark3/jars/leveldbjni-all-1.8.jar /home/jslorrma/.spark3/jars/libfb303-0.9.3.jar /home/jslorrma/.spark3/jars/libthrift-0.12.0.jar /home/jslorrma/.spark3/jars/log4j-1.2.17.jar /home/jslorrma/.spark3/jars/logging-interceptor-3.12.12.jar /home/jslorrma/.spark3/jars/lz4-java-1.7.1.jar /home/jslorrma/.spark3/jars/machinist_2.12-0.6.8.jar /home/jslorrma/.spark3/jars/macro-compat_2.12-1.1.1.jar /home/jslorrma/.spark3/jars/mesos-1.4.0-shaded-protobuf.jar /home/jslorrma/.spark3/jars/metrics-core-4.1.1.jar /home/jslorrma/.spark3/jars/metrics-graphite-4.1.1.jar /home/jslorrma/.spark3/jars/metrics-jmx-4.1.1.jar /home/jslorrma/.spark3/jars/metrics-json-4.1.1.jar /home/jslorrma/.spark3/jars/metrics-jvm-4.1.1.jar /home/jslorrma/.spark3/jars/minlog-1.3.0.jar /home/jslorrma/.spark3/jars/netty-all-4.1.51.Final.jar /home/jslorrma/.spark3/jars/nimbus-jose-jwt-4.41.1.jar /home/jslorrma/.spark3/jars/objenesis-2.6.jar /home/jslorrma/.spark3/jars/okhttp-2.7.5.jar /home/jslorrma/.spark3/jars/okhttp-3.12.12.jar /home/jslorrma/.spark3/jars/okio-1.14.0.jar /home/jslorrma/.spark3/jars/opencsv-2.3.jar /home/jslorrma/.spark3/jars/orc-core-1.5.12.jar /home/jslorrma/.spark3/jars/orc-mapreduce-1.5.12.jar /home/jslorrma/.spark3/jars/orc-shims-1.5.12.jar /home/jslorrma/.spark3/jars/oro-2.0.8.jar /home/jslorrma/.spark3/jars/osgi-resource-locator-1.0.3.jar /home/jslorrma/.spark3/jars/paranamer-2.8.jar /home/jslorrma/.spark3/jars/parquet-column-1.10.1.jar /home/jslorrma/.spark3/jars/parquet-common-1.10.1.jar /home/jslorrma/.spark3/jars/parquet-encoding-1.10.1.jar /home/jslorrma/.spark3/jars/parquet-format-2.4.0.jar /home/jslorrma/.spark3/jars/parquet-hadoop-1.10.1.jar /home/jslorrma/.spark3/jars/parquet-jackson-1.10.1.jar /home/jslorrma/.spark3/jars/protobuf-java-2.5.0.jar /home/jslorrma/.spark3/jars/py4j-0.10.9.jar /home/jslorrma/.spark3/jars/pyrolite-4.30.jar /home/jslorrma/.spark3/jars/re2j-1.1.jar /home/jslorrma/.spark3/jars/scala-collection-compat_2.12-2.1.1.jar /home/jslorrma/.spark3/jars/scala-compiler-2.12.10.jar /home/jslorrma/.spark3/jars/scala-library-2.12.10.jar /home/jslorrma/.spark3/jars/scala-parser-combinators_2.12-1.1.2.jar /home/jslorrma/.spark3/jars/scala-reflect-2.12.10.jar /home/jslorrma/.spark3/jars/scala-xml_2.12-1.2.0.jar /home/jslorrma/.spark3/jars/shapeless_2.12-2.3.3.jar /home/jslorrma/.spark3/jars/shims-0.9.0.jar /home/jslorrma/.spark3/jars/slf4j-api-1.7.30.jar /home/jslorrma/.spark3/jars/slf4j-log4j12-1.7.30.jar /home/jslorrma/.spark3/jars/snakeyaml-1.24.jar /home/jslorrma/.spark3/jars/snappy-java-1.1.8.2.jar /home/jslorrma/.spark3/jars/spark-avro_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-catalyst_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-core_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-graphx_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-hive-thriftserver_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-hive_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-kubernetes_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-kvstore_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-launcher_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-mesos_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-mllib-local_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-mllib_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-network-common_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-network-shuffle_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-repl_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-sketch_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-sql_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-streaming_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-tags_2.12-3.1.2-tests.jar /home/jslorrma/.spark3/jars/spark-tags_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-unsafe_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spark-yarn_2.12-3.1.2.jar /home/jslorrma/.spark3/jars/spire-macros_2.12-0.17.0-M1.jar /home/jslorrma/.spark3/jars/spire-platform_2.12-0.17.0-M1.jar /home/jslorrma/.spark3/jars/spire-util_2.12-0.17.0-M1.jar /home/jslorrma/.spark3/jars/spire_2.12-0.17.0-M1.jar /home/jslorrma/.spark3/jars/stax-api-1.0.1.jar /home/jslorrma/.spark3/jars/stax2-api-3.1.4.jar /home/jslorrma/.spark3/jars/stream-2.9.6.jar /home/jslorrma/.spark3/jars/super-csv-2.2.0.jar /home/jslorrma/.spark3/jars/threeten-extra-1.5.0.jar /home/jslorrma/.spark3/jars/token-provider-1.0.1.jar /home/jslorrma/.spark3/jars/transaction-api-1.1.jar /home/jslorrma/.spark3/jars/univocity-parsers-2.9.1.jar /home/jslorrma/.spark3/jars/velocity-1.5.jar /home/jslorrma/.spark3/jars/woodstox-core-5.0.3.jar /home/jslorrma/.spark3/jars/xbean-asm7-shaded-4.15.jar /home/jslorrma/.spark3/jars/xz-1.5.jar /home/jslorrma/.spark3/jars/zjsonpatch-0.3.0.jar /home/jslorrma/.spark3/jars/zookeeper-3.4.14.jar /home/jslorrma/.spark3/jars/zstd-jni-1.4.8-1.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/accessors-smart-1.2.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/accessors-smart.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/asm-5.0.4.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/asm.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/avro.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/aws-java-sdk-bundle-1.11.271.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/aws-java-sdk-bundle.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/azure-data-lake-store-sdk-2.2.9.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/azure-data-lake-store-sdk.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-beanutils-1.9.4.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-beanutils.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-cli-1.2.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-cli.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-codec-1.11.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-codec.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-collections-3.2.2.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-collections.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-compress-1.19.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-compress.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-configuration2-2.1.1.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-configuration2.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-io-2.6.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-io.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-lang-2.6.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-lang.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-lang3-3.7.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-lang3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-logging-1.1.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-logging.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-math3-3.1.1.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-math3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-net-3.1.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/commons-net.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/curator-client-2.12.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/curator-client.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/curator-framework-2.12.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/curator-framework.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/curator-recipes-2.12.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/curator-recipes.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/gson-2.2.4.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/gson.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/guava-11.0.2.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/guava.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-annotations-3.0.0-cdh6.3.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-annotations.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-auth-3.0.0-cdh6.3.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-auth.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-aws-3.0.0-cdh6.3.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-aws.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-azure-3.0.0-cdh6.3.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-azure-datalake-3.0.0-cdh6.3.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-azure-datalake.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-azure.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-common-3.0.0-cdh6.3.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-common.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-hdfs-client-3.0.0-cdh6.3.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-hdfs-client.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-mapreduce-client-common-3.0.0-cdh6.3.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-mapreduce-client-common.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-mapreduce-client-core-3.0.0-cdh6.3.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-mapreduce-client-core.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-mapreduce-client-jobclient-3.0.0-cdh6.3.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-mapreduce-client-jobclient.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-yarn-api-3.0.0-cdh6.3.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-yarn-api.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-yarn-client-3.0.0-cdh6.3.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-yarn-client.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-yarn-common-3.0.0-cdh6.3.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/hadoop-yarn-common.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/htrace-core4-4.1.0-incubating.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/htrace-core4.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/httpclient-4.5.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/httpclient.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/httpcore-4.4.6.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/httpcore.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jackson-annotations-2.9.10.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jackson-annotations.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jackson-core-2.9.10.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jackson-core-asl-1.9.13.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jackson-core-asl.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jackson-core.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jackson-databind-2.9.10.1.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jackson-databind.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jackson-jaxrs-base-2.9.10.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jackson-jaxrs-base.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jackson-jaxrs-json-provider-2.9.10.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jackson-jaxrs-json-provider.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jackson-mapper-asl-1.9.13-cloudera.1.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jackson-mapper-asl.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jackson-module-jaxb-annotations-2.9.10.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jackson-module-jaxb-annotations.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/javax.activation-api-1.2.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/javax.activation-api.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/javax.servlet-api-3.1.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/javax.servlet-api.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jaxb-api-2.2.11.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jaxb-api.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jcip-annotations-1.0-1.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jcip-annotations.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jersey-client-1.19.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jersey-client.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jersey-core-1.19.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jersey-core.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jersey-servlet-1.19.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jersey-servlet.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jetty-security-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jetty-security.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jetty-servlet-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jetty-servlet.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jetty-util-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jetty-util.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jetty-webapp-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jetty-webapp.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jetty-xml-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jetty-xml.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/json-smart-2.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/json-smart.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jsp-api-2.1.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jsp-api.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jsr305-3.0.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jsr305.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jsr311-api-1.1.1.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/jsr311-api.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-admin-1.0.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-admin.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-client-1.0.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-client.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-common-1.0.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-common.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-core-1.0.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-core.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-crypto-1.0.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-crypto.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-identity-1.0.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-identity.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-server-1.0.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-server.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-simplekdc-1.0.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-simplekdc.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-util-1.0.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerb-util.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerby-asn1-1.0.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerby-asn1.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerby-config-1.0.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerby-config.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerby-pkix-1.0.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerby-pkix.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerby-util-1.0.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerby-util.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerby-xdr-1.0.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/kerby-xdr.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/log4j-1.2.17.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/log4j.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/nimbus-jose-jwt-4.41.1.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/nimbus-jose-jwt.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/okhttp-2.7.5.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/okhttp.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/okio-1.6.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/okio.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/paranamer-2.8.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/paranamer.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/protobuf-java-2.5.0.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/protobuf-java.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/re2j-1.1.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/re2j.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/slf4j-api-1.7.25.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/slf4j-api.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/snappy-java-1.1.4.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/snappy-java.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/stax2-api-3.1.4.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/stax2-api.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/wildfly-openssl-1.0.4.Final.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/wildfly-openssl.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/woodstox-core-5.0.3.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/woodstox-core.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/xz-1.6.jar /opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p4744.12781922/lib/hadoop/client/xz.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/./ /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/.//hadoop-hdfs-3.0.0-cdh6.3.4-tests.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/.//hadoop-hdfs-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/.//hadoop-hdfs-client-3.0.0-cdh6.3.4-tests.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/.//hadoop-hdfs-client-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/.//hadoop-hdfs-client-tests.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/.//hadoop-hdfs-client.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/.//hadoop-hdfs-httpfs-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/.//hadoop-hdfs-httpfs.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/.//hadoop-hdfs-native-client-3.0.0-cdh6.3.4-tests.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/.//hadoop-hdfs-native-client-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/.//hadoop-hdfs-native-client-tests.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/.//hadoop-hdfs-native-client.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/.//hadoop-hdfs-nfs-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/.//hadoop-hdfs-nfs.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/.//hadoop-hdfs-tests.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/.//hadoop-hdfs.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/accessors-smart-1.2.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/asm-5.0.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/audience-annotations-0.5.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/avro-1.8.2-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/commons-beanutils-1.9.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/commons-cli-1.2.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/commons-codec-1.11.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/commons-collections-3.2.2.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/commons-compress-1.19.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/commons-configuration2-2.1.1.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/commons-io-2.6.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/commons-lang-2.6.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/commons-lang3-3.7.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/commons-math3-3.1.1.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/commons-net-3.1.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/curator-client-2.12.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/curator-framework-2.12.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/curator-recipes-2.12.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/gson-2.2.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/guava-11.0.2.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/htrace-core4-4.1.0-incubating.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/httpclient-4.5.3.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/httpcore-4.4.6.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jackson-annotations-2.9.10.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jackson-core-2.9.10.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jackson-core-asl-1.9.13.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jackson-databind-2.9.10.6.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jackson-jaxrs-1.9.13.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.9.13-cloudera.1.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jackson-xc-1.9.13.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/javax.activation-api-1.2.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/javax.servlet-api-3.1.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jaxb-api-2.2.11.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jaxb-impl-2.2.3-1.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jcip-annotations-1.0-1.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jersey-core-1.19.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jersey-json-1.19.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jersey-server-1.19.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jersey-servlet-1.19.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jettison-1.1.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jetty-http-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jetty-io-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jetty-security-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jetty-server-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jetty-servlet-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jetty-util-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jetty-util-ajax-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jetty-webapp-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jetty-xml-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jsch-0.1.54.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/json-simple-1.1.1.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/json-smart-2.3.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/jsr311-api-1.1.1.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/kerb-admin-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/kerb-client-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/kerb-common-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/kerb-core-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/kerb-crypto-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/kerb-identity-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/kerb-server-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/kerb-simplekdc-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/kerb-util-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/kerby-asn1-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/kerby-config-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/kerby-pkix-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/kerby-util-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/kerby-xdr-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/log4j-1.2.17.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/netty-3.10.6.Final.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/nimbus-jose-jwt-7.9.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/okhttp-2.7.5.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/okio-1.6.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/paranamer-2.8.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/re2j-1.1.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/snappy-java-1.1.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/stax2-api-3.1.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/woodstox-core-5.0.3.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/xz-1.6.jar /opt/cloudera/parcels/CDH/lib/hadoop-hdfs/lib/zookeeper-3.4.5-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//aliyun-sdk-oss-2.8.3.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//azure-keyvault-core-0.8.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//azure-storage-5.4.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-aliyun-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-aliyun.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-archive-logs-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-archive-logs.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-archives-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-archives.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-datajoin-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-datajoin.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-distcp-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-distcp.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-extras-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-extras.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-gridmix-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-gridmix.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-kafka-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-kafka.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-3.0.0-cdh6.3.4-tests.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-tests.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-nativetask-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-nativetask.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-uploader-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-uploader.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-openstack-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-openstack.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-resourceestimator-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-resourceestimator.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-rumen-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-rumen.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-sls-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-sls.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-streaming-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-streaming.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jdom-1.1.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//kafka-clients-2.2.1-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//lz4-java-1.5.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//ojalgo-43.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//zstd-jni-1.3.8-1.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-api-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-api.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-client-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-client.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-common-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-common.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-registry-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-registry.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-applicationhistoryservice-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-applicationhistoryservice.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-common-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-common.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-router-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-router.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-sharedcachemanager-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-sharedcachemanager.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-tests-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-tests.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-timeline-pluginstorage-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-timeline-pluginstorage.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/HikariCP-java7-2.4.12.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/aopalliance-1.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/bcpkix-jdk15on-1.60.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/bcprov-jdk15on-1.60.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/ehcache-3.3.1.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/fst-2.50.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/guice-4.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/guice-servlet-4.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/jackson-jaxrs-base-2.9.10.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/jackson-jaxrs-json-provider-2.9.10.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/jackson-module-jaxb-annotations-2.9.10.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/java-util-1.9.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/javax.inject-1.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/jersey-client-1.19.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/jersey-guice-1.19.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/json-io-2.5.1.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/metrics-core-3.0.1.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/mssql-jdbc-6.2.1.jre7.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/objenesis-1.0.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/spark-2.4.0-cdh6.3.4-yarn-shuffle.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-annotations-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-annotations.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-auth-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-auth.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-aws-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-aws.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-azure-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-azure-datalake-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-azure-datalake.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-azure.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-common-3.0.0-cdh6.3.4-tests.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-common-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-common-tests.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-common.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-kms-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-kms.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-nfs-3.0.0-cdh6.3.4.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//hadoop-nfs.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-avro.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-cascading.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-cascading3.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-column.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-common.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-encoding.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-format-javadoc.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-format-sources.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-format.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-generator.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-hadoop-bundle.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-hadoop.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-jackson.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-pig-bundle.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-pig.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-protobuf.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-scala_2.11.jar /opt/cloudera/parcels/CDH/lib/hadoop/.//parquet-thrift.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/accessors-smart-1.2.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/asm-5.0.4.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/audience-annotations-0.5.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/avro.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/aws-java-sdk-bundle-1.11.563.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/azure-data-lake-store-sdk-2.2.9.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/commons-beanutils-1.9.4.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/commons-cli-1.2.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/commons-codec-1.11.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/commons-collections-3.2.2.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/commons-compress-1.19.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/commons-configuration2-2.1.1.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/commons-io-2.6.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/commons-lang-2.6.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/commons-lang3-3.7.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/commons-logging-1.1.3.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/commons-math3-3.1.1.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/commons-net-3.1.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/curator-client-2.12.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/curator-framework-2.12.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/curator-recipes-2.12.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/gson-2.2.4.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/guava-11.0.2.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/htrace-core4-4.1.0-incubating.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/httpclient-4.5.3.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/httpcore-4.4.6.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jackson-annotations-2.9.10.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jackson-core-2.9.10.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jackson-core-asl-1.9.13.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jackson-databind-2.9.10.6.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jackson-jaxrs-1.9.13.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jackson-mapper-asl-1.9.13-cloudera.1.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jackson-xc-1.9.13.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/javax.activation-api-1.2.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/javax.servlet-api-3.1.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jaxb-api-2.2.11.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jcip-annotations-1.0-1.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jersey-core-1.19.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jersey-json-1.19.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jersey-server-1.19.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jersey-servlet-1.19.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jettison-1.1.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jetty-http-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jetty-io-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jetty-security-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jetty-server-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jetty-servlet-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jetty-util-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jetty-webapp-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jetty-xml-9.3.25.v20180904.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jsch-0.1.54.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/json-smart-2.3.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jsp-api-2.1.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jsr305-3.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jsr311-api-1.1.1.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/jul-to-slf4j-1.7.25.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/kerb-admin-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/kerb-client-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/kerb-common-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/kerb-core-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/kerb-crypto-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/kerb-identity-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/kerb-server-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/kerb-simplekdc-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/kerb-util-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/kerby-asn1-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/kerby-config-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/kerby-pkix-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/kerby-util-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/kerby-xdr-1.0.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/log4j-1.2.17.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/log4j-api-2.8.2.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/log4j-core-2.8.2.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/logredactor-2.0.7.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/metrics-core-3.0.1.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/netty-3.10.6.Final.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/nimbus-jose-jwt-7.9.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/paranamer-2.8.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/protobuf-java-2.5.0.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/re2j-1.1.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/slf4j-api-1.7.25.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/slf4j-log4j12-1.7.25.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/slf4j-log4j12.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/snappy-java-1.1.4.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/stax2-api-3.1.4.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/wildfly-openssl-1.0.4.Final.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/woodstox-core-5.0.3.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/xz-1.6.jar /opt/cloudera/parcels/CDH/lib/hadoop/lib/zookeeper.jar ```
SparkContext config: ``` ('spark.eventLog.enabled', 'true'), ('spark.driver.cores', '4'), ('spark.pyspark.driver.python', '/home/jslorrma/.miniconda3/envs/pyspark3/bin/python'), ('spark.network.crypto.enabled', 'true'), ('spark.yarn.appMasterEnv.MKL_NUM_THREADS', '1'), ('spark.executor.cores', '5') ('spark.serializer', 'org.apache.spark.serializer.KryoSerializer'), ('spark.driverEnv.PYSPARK_DRIVER_PYTHON', '/home/jslorrma/.miniconda3/envs/pyspark3/bin/python'), ('spark.ui.port', '20107'), ('spark.sql.parquet.compression.codec', 'snappy'), ('spark.blockManager.port', '20200'), ('spark.master', 'local[*]'), ('spark.sql.files.openCostInBytes', '1'), ('spark.sql.execution.arrow.pyspark.fallback.enabled', 'false'), ('spark.sql.execution.arrow.pyspark.enabled', 'true'), ('spark.lineage.enabled', 'false'), ('spark.app.id', 'local-1629214593490'), ('spark.executorEnv.MKL_NUM_THREADS', '1'), ('spark.ui.enabled', 'true'), ('spark.executor.id', 'driver'), ('spark.dynamicAllocation.schedulerBacklogTimeout', '1'), ('spark.driver.memory', '20g'), ('spark.driver.log.persistToDfs.enabled', 'true'), ('spark.lineage.log.dir', '/opt/cloudera/log/spark/lineage'), ('spark.app.startTime', '1629214591703'), ('spark.executorEnv.OPENBLAS_NUM_THREADS', '1'), ('spark.dynamicAllocation.executorIdleTimeout', '600'), ('spark.sql.files.maxPartitionBytes', '1073741824'), ('spark.broadcast.port', '20065'), ('spark.executor.memoryOverhead', '3g'), ('spark.driver.port', '20000'), ('spark.yarn.appMasterEnv.PYSPARK_PYTHON', '/home/jslorrma/.miniconda3/bin/pyton'), ('spark.executorEnv.PYSPARK_PYTHON', '/home/jslorrma/.miniconda3/envs/pyspark3/bin/python'), ('spark.port.maxRetries', '48'), ('spark.pyspark.python', '/home/jslorrma/.miniconda3/envs/pyspark3/bin/python'), ('spark.app.name','hudi test'), ('spark.appMasterEnv.PYSPARK_DRIVER_PYTHON', '/home/jslorrma/.miniconda3/envs/pyspark3/bin/python'), ('spark.driver.memoryOverhead', '8g'), ('spark.io.encryption.enabled', 'true'), ('spark.serializer.objectStreamReset', '100'), ('spark.network.sasl.serverAlwaysEncrypt', 'true'), ('spark.submit.deployMode', 'client'), ('spark.replClassServer.port', '20086'), ('spark.shuffle.service.enabled', 'true'), ('spark.yarn.appMasterEnv.OPENBLAS_NUM_THREADS', '1'), ('spark.shuffle.service.port', '7337'), ('spark.executor.memory', '6g'), ('spark.default.parallelism', '5'), ('spark.rdd.compress', 'True'), ('spark.yarn.appMasterEnv.PYSPARK_DRIVER_PYTHON', '/home/jslorrma/.miniconda3/bin/python'), ('spark.dynamicAllocation.minExecutors', '0'), ('spark.submit.pyFiles', ''), ('spark.dynamicAllocation.enabled', 'true'), ('spark.ui.showConsoleProgress', 'true'), ('spark.driver.log.dfsDir', '/user/spark/driverLogs') ```

Stacktrace

Add the stacktrace of the error.

Thanks for your help already in advance!!

jslorrma commented 3 years ago

I tried today instead of running the code above locally (.set("spark.master", "local[*]")) to run it on our Hadoop cluster via YARN (.set("spark.master", "yarn")). Everything works as expected and without any error on the cluster. So the error seems to have something to do with my local Spark / Hadoop installation and configuration. Via Google I found this link which might help in this context (java.lang.UnsatisfiedLinkError), although it is for OS=Windows.

So, @dev-Team / @support-Team, should I close this issue, as it seems to be not related to Hudi or keep it open for further discussions?

nsivabalan commented 3 years ago

yeah, we can close the issue out. Once you find the root cause and fix it, would appreciate if you can add an entry here to help others in the community