search

JohnSnowLabs / spark-nlp

State of the Art Natural Language Processing
https://sparknlp.org/
Apache License 2.0
3.86k stars 711 forks source link

Tensorflow lib core dumped #996

Closed miloradtrninic closed 3 years ago

miloradtrninic commented 4 years ago

When I try to use pretrained model I get core dumped. Error is below. 

2020-08-05 14:35:59 INFO  HadoopRDD:54 - Input split: hdfs://namenode:9000/models/recognize_entities_dl/stages/4_NerDLModel_d4424c9af5f4/fields/datasetParams/part-00011:0+2831
2020-08-05 14:35:59 INFO  Executor:54 - Finished task 4.0 in stage 16.0 (TID 32). 765 bytes result sent to driver
2020-08-05 14:35:59 INFO  TaskSetManager:54 - Finished task 4.0 in stage 16.0 (TID 32) in 38 ms on localhost (executor driver) (5/7)
2020-08-05 14:35:59 INFO  Executor:54 - Finished task 5.0 in stage 16.0 (TID 33). 765 bytes result sent to driver
2020-08-05 14:35:59 INFO  TaskSetManager:54 - Finished task 5.0 in stage 16.0 (TID 33) in 47 ms on localhost (executor driver) (6/7)
2020-08-05 14:35:59 INFO  Executor:54 - Finished task 6.0 in stage 16.0 (TID 34). 2146 bytes result sent to driver
2020-08-05 14:35:59 INFO  TaskSetManager:54 - Finished task 6.0 in stage 16.0 (TID 34) in 55 ms on localhost (executor driver) (7/7)
2020-08-05 14:35:59 INFO  TaskSchedulerImpl:54 - Removed TaskSet 16.0, whose tasks have all completed, from pool 
2020-08-05 14:35:59 INFO  DAGScheduler:54 - ResultStage 16 (first at Feature.scala:120) finished in 0.110 s
2020-08-05 14:35:59 INFO  DAGScheduler:54 - Job 16 finished: first at Feature.scala:120, took 0.119676 s
2020-08-05 14:35:59 INFO  MemoryStore:54 - Block broadcast_31 stored as values in memory (estimated size 8.4 KB, free 361.2 MB)
2020-08-05 14:35:59 INFO  MemoryStore:54 - Block broadcast_31_piece0 stored as bytes in memory (estimated size 440.0 B, free 361.2 MB)
2020-08-05 14:35:59 INFO  BlockManagerInfo:54 - Added broadcast_31_piece0 in memory on 82a79ae5305b:45455 (size: 440.0 B, free: 365.8 MB)
2020-08-05 14:35:59 INFO  SparkContext:54 - Created broadcast 31 from broadcast at Feature.scala:87
\#
\# A fatal error has been detected by the Java Runtime Environment:
\#
\#  SIGILL (0x4) at pc=0x00007f2dae59ada9, pid=846, tid=0x00007f2e5dad5700
\#
\# JRE version: OpenJDK Runtime Environment (8.0_171-b11) (build 1.8.0_171-8u171-b11-1~bpo8+1-b11)
\# Java VM: OpenJDK 64-Bit Server VM (25.171-b11 mixed mode linux-amd64 compressed oops)
\# Problematic frame:
\# C  [libtensorflow_framework.so.1+0x744da9]  _GLOBAL__sub_I_loader.cc+0x99
\#
\# Core dump written. Default location: //core or core.846
\#
\# An error report file with more information is saved as:
\# //hs_err_pid846.log
\#
\# If you would like to submit a bug report, please visit:
\#   http://bugreport.java.com/bugreport/crash.jsp
\# The crash happened outside the Java Virtual Machine in native code.
\# See problematic frame for where to report the bug.

Steps to Reproduce

  1. Clone the repo https://github.com/miloradtrninic/entity/
  2. Run docker compose from cloned directory
  3. Download recognize_entities_dl for offline usage (recognize_entities_dl_en_2.4.3_2.4_1584626752821)
  4. Unzip on local computer model
  5. docker exec namenode mkdir models
  6. docker cp recognize_entities_dl/ namenode:/models/recognize_entities_dl
  7. docker exec namenode hdfs dfs -mkdir /models
  8. docker exec namenode hdfs dfs -put /models/recognize_entities_dl/ /models/
  9. Run " ./submit.sh b=1 d=1 e=1 a=/ " it will give 1GB to driver and executors and build the project with sbt assembly.

Context

I am getting core dumped on simple execution of the spark nlp framework.

It seams a lot like #923 but I think I provided reproducible environment.

This issue along with #985 is blocking me completely from using the library and proceeding with my masters thesis. Until it is fixed can you provide some docker images you know it is working on? I have this same issue when I use offline models for the spark-nlp starter project in this environment.

Your Environment

maziyarpanahi commented 4 years ago

What is the exact operating system? (Your linux distribution)

miloradtrninic commented 4 years ago

Debian GNU/Linux 8 (jessie)

maziyarpanahi commented 4 years ago

Thank you, is there a way to share hs_err_pid846.log file that has the core dump with us?

miloradtrninic commented 4 years ago

Here is the log https://github.com/miloradtrninic/entity/blob/master/dump_log.txt

maziyarpanahi commented 4 years ago

Could you please try this in your build.sbt:


import com.typesafe.sbt.packager.archetypes.JavaAppPackaging

enablePlugins(JavaServerAppPackaging)
enablePlugins(JavaAppPackaging)

libraryDependencies ++= {
  val sparkVer = "2.4.4"
  val sparkNLP = "2.5.4"
  Seq(
    "org.apache.spark" %% "spark-core" % sparkVer % "provided",
    "org.apache.spark" %% "spark-mllib" % sparkVer % "provided",
    "com.johnsnowlabs.nlp" %% "spark-nlp" % sparkNLP % "provided"
  )
}

//conflictManager := ConflictManager.strict

assemblyMergeStrategy in assembly := {
  case PathList("META-INF", xs @ _*) => MergeStrategy.discard
  case x if x.startsWith("NativeLibrary") => MergeStrategy.last
  case x if x.startsWith("aws") => MergeStrategy.last
  case _ => MergeStrategy.last
}
////
assemblyExcludedJars in assembly := {
  val cp = (fullClasspath in assembly).value
  cp filter {
    j => {
        j.data.getName.startsWith("spark-nlp")
    }
  }
}

and then run sbt-submit as follow:

spark-submit --packages com.johnsnowlabs.nlp:spark-nlp_2.11:2.5.4 --driver-memory 8g --class "Main" /spark-nlp-starter/target/scala-2.11/spark-nlp-starter-assembly-0.1.jar
miloradtrninic commented 4 years ago

It seams to me that you have placed a comment in wrong ticket Did you mean to comment on #985?

But I tried out with these changes to fix tensorflow lib issue without success, got the same issue. But when I did not use "provided" for any of the dependencies error changed to 

2020-08-05 23:39:42 INFO  DAGScheduler:54 - ResultStage 16 (first at Feature.scala:120) finished in 0.074 s
2020-08-05 23:39:42 INFO  DAGScheduler:54 - Job 16 finished: first at Feature.scala:120, took 0.078495 s
2020-08-05 23:39:42 INFO  MemoryStore:54 - Block broadcast_31 stored as values in memory (estimated size 8.4 KB, free 362.9 MB)
2020-08-05 23:39:42 INFO  MemoryStore:54 - Block broadcast_31_piece0 stored as bytes in memory (estimated size 440.0 B, free 362.9 MB)
2020-08-05 23:39:42 INFO  BlockManagerInfo:54 - Added broadcast_31_piece0 in memory on 3965c6cd1f0b:43077 (size: 440.0 B, free: 366.0 MB)
2020-08-05 23:39:42 INFO  SparkContext:54 - Created broadcast 31 from broadcast at Feature.scala:87
Exception in thread "main" java.lang.UnsatisfiedLinkError: /tmp/tensorflow_native_libraries-1596670782715-0/libtensorflow_jni.so: libtensorflow_framework.so.1: cannot open shared object file: No such file or directory
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
    at java.lang.Runtime.load0(Runtime.java:809)
    at java.lang.System.load(System.java:1086)
    at org.tensorflow.NativeLibrary.load(NativeLibrary.java:101)
    at org.tensorflow.TensorFlow.init(TensorFlow.java:67)
    at org.tensorflow.TensorFlow.<clinit>(TensorFlow.java:82)
    at org.tensorflow.Graph.<clinit>(Graph.java:479)
    at com.johnsnowlabs.ml.tensorflow.TensorflowWrapper$.readGraph(TensorflowWrapper.scala:204)
    at com.johnsnowlabs.ml.tensorflow.TensorflowWrapper$.read(TensorflowWrapper.scala:249)
    at com.johnsnowlabs.ml.tensorflow.ReadTensorflowModel$class.readTensorflowModel(TensorflowSerializeModel.scala:97)
    at com.johnsnowlabs.nlp.annotators.ner.dl.NerDLModel$.readTensorflowModel(NerDLModel.scala:144)
    at com.johnsnowlabs.nlp.annotators.ner.dl.ReadsNERGraph$class.readNerGraph(NerDLModel.scala:124)
    at com.johnsnowlabs.nlp.annotators.ner.dl.NerDLModel$.readNerGraph(NerDLModel.scala:144)
    at com.johnsnowlabs.nlp.annotators.ner.dl.ReadsNERGraph$$anonfun$2.apply(NerDLModel.scala:128)
    at com.johnsnowlabs.nlp.annotators.ner.dl.ReadsNERGraph$$anonfun$2.apply(NerDLModel.scala:128)
    at com.johnsnowlabs.nlp.ParamsAndFeaturesReadable$$anonfun$com$johnsnowlabs$nlp$ParamsAndFeaturesReadable$$onRead$1.apply(ParamsAndFeaturesReadable.scala:31)
    at com.johnsnowlabs.nlp.ParamsAndFeaturesReadable$$anonfun$com$johnsnowlabs$nlp$ParamsAndFeaturesReadable$$onRead$1.apply(ParamsAndFeaturesReadable.scala:30)
    at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
    at com.johnsnowlabs.nlp.ParamsAndFeaturesReadable$class.com$johnsnowlabs$nlp$ParamsAndFeaturesReadable$$onRead(ParamsAndFeaturesReadable.scala:30)
    at com.johnsnowlabs.nlp.ParamsAndFeaturesReadable$$anonfun$read$1.apply(ParamsAndFeaturesReadable.scala:41)
    at com.johnsnowlabs.nlp.ParamsAndFeaturesReadable$$anonfun$read$1.apply(ParamsAndFeaturesReadable.scala:41)
    at com.johnsnowlabs.nlp.FeaturesReader.load(ParamsAndFeaturesReadable.scala:19)
    at com.johnsnowlabs.nlp.FeaturesReader.load(ParamsAndFeaturesReadable.scala:8)
    at org.apache.spark.ml.util.DefaultParamsReader$.loadParamsInstance(ReadWrite.scala:652)
    at org.apache.spark.ml.Pipeline$SharedReadWrite$$anonfun$4.apply(Pipeline.scala:274)
    at org.apache.spark.ml.Pipeline$SharedReadWrite$$anonfun$4.apply(Pipeline.scala:272)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
    at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
    at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186)
    at org.apache.spark.ml.Pipeline$SharedReadWrite$.load(Pipeline.scala:272)
    at org.apache.spark.ml.PipelineModel$PipelineModelReader.load(Pipeline.scala:348)
    at org.apache.spark.ml.PipelineModel$PipelineModelReader.load(Pipeline.scala:342)
    at org.apache.spark.ml.util.MLReadable$class.load(ReadWrite.scala:380)
    at org.apache.spark.ml.PipelineModel$.load(Pipeline.scala:332)
    at com.johnsnowlabs.nlp.pretrained.PretrainedPipeline.<init>(PretrainedPipeline.scala:28)
    at com.synechron.entity.entityresolution.ResolutionMain$.main(ResolutionMain.scala:20)
    at com.synechron.entity.entityresolution.ResolutionMain.main(ResolutionMain.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
    at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
    at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
    at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
    at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
    at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
maziyarpanahi commented 4 years ago

No, it's for here and from now on we continue here since we found out why the models cannot be downloaded and there is a PR for that. The workaround until the next release is to add provide and use --packages in spark-submit.

Now on to TensorFlow. There can be two reasons for that, first, the OS is not supported or second, we failed to shade the file we override in the TensorFlow dependency.

Could you please use this jar with the following command:

spark-submit --packages com.johnsnowlabs.nlp:spark-nlp_2.11:2.5.4 --driver-memory 8g --class "Main" spark-nlp-starter-fix.jar

spark-nlp-starter-fix.jar.zip

And if it fails again, could you please do the same thing but this time change your Docker image to Ubuntu instead of Debian?

miloradtrninic commented 4 years ago

I have tried out with the jar you provided I had to use 4GB because I have 8GB in total. Plus I had to add --packages org.apache.spark:spark-hive_2.11:2.4.0 because of the .enableHiveSupport() Also note that my spark is 2.4.0.

This is the error with the Spark NLP jar:

docker exec spark-master /spark/bin/spark-submit --packages org.apache.spark:spark-hive_2.11:2.4.0,com.johnsnowlabs.nlp:spark-nlp_2.11:2.5.4 --driver-memory 4g --class "Main" /deploy/spark-nlp-starter-fix.jar

Ivy Default Cache set to: /root/.ivy2/cache
The jars for the packages stored in: /root/.ivy2/jars
:: loading settings :: url = jar:file:/spark/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
org.apache.spark#spark-hive_2.11 added as a dependency
com.johnsnowlabs.nlp#spark-nlp_2.11 added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent-d3ee78bf-b90d-4418-bbd7-02772086063f;1.0
    confs: [default]
    found org.apache.spark#spark-hive_2.11;2.4.0 in central
    found com.twitter#parquet-hadoop-bundle;1.6.0 in central
    found org.spark-project.hive#hive-exec;1.2.1.spark2 in central
    found commons-io#commons-io;2.4 in central
    found org.apache.commons#commons-lang3;3.5 in central
    found commons-lang#commons-lang;2.6 in central
    found javolution#javolution;5.5.1 in central
    found log4j#apache-log4j-extras;1.2.17 in central
    found org.antlr#antlr-runtime;3.4 in central
    found org.antlr#stringtemplate;3.2.1 in central
    found antlr#antlr;2.7.7 in central
    found org.antlr#ST4;4.0.4 in central
    found org.apache.avro#avro;1.8.2 in central
    found org.codehaus.jackson#jackson-core-asl;1.9.13 in central
    found org.codehaus.jackson#jackson-mapper-asl;1.9.13 in central
    found com.thoughtworks.paranamer#paranamer;2.8 in central
    found org.xerial.snappy#snappy-java;1.1.7.1 in central
    found org.apache.commons#commons-compress;1.4.1 in central
    found org.tukaani#xz;1.5 in central
    found org.slf4j#slf4j-api;1.7.16 in central
    found org.apache.ivy#ivy;2.4.0 in central
    found org.jodd#jodd-core;3.5.2 in central
    found org.datanucleus#datanucleus-core;3.2.10 in central
    found org.apache.calcite#calcite-avatica;1.2.0-incubating in central
    found com.googlecode.javaewah#JavaEWAH;0.3.2 in central
    found org.iq80.snappy#snappy;0.2 in central
    found stax#stax-api;1.0.1 in central
    found net.sf.opencsv#opencsv;2.3 in central
    found org.spark-project.hive#hive-metastore;1.2.1.spark2 in central
    found com.jolbox#bonecp;0.8.0.RELEASE in central
    found commons-cli#commons-cli;1.2 in central
    found commons-logging#commons-logging;1.1.3 in central
    found org.apache.derby#derby;10.12.1.1 in central
    found org.datanucleus#datanucleus-api-jdo;3.2.6 in central
    found org.datanucleus#datanucleus-rdbms;3.2.9 in central
    found commons-pool#commons-pool;1.5.4 in central
    found commons-dbcp#commons-dbcp;1.4 in central
    found javax.jdo#jdo-api;3.0.1 in central
    found javax.transaction#jta;1.1 in central
    found org.apache.avro#avro-mapred;1.8.2 in central
    found org.apache.avro#avro-ipc;1.8.2 in central
    found commons-codec#commons-codec;1.10 in central
    found commons-httpclient#commons-httpclient;3.1 in central
    found org.apache.calcite#calcite-core;1.2.0-incubating in central
    found org.apache.calcite#calcite-linq4j;1.2.0-incubating in central
    found net.hydromatic#eigenbase-properties;1.1.5 in central
    found org.apache.httpcomponents#httpclient;4.5.6 in central
    found org.apache.httpcomponents#httpcore;4.4.10 in central
    found commons-logging#commons-logging;1.2 in central
    found joda-time#joda-time;2.9.3 in central
    found com.google.code.findbugs#jsr305;1.3.9 in central
    found org.apache.thrift#libthrift;0.9.3 in central
    found org.apache.thrift#libfb303;0.9.3 in central
    found org.spark-project.spark#unused;1.0.0 in central
    found com.johnsnowlabs.nlp#spark-nlp_2.11;2.5.4 in central
    found com.typesafe#config;1.3.0 in central
    found org.rocksdb#rocksdbjni;6.5.3 in central
    found org.apache.hadoop#hadoop-aws;3.2.0 in central
    found com.amazonaws#aws-java-sdk-core;1.11.603 in central
    found org.apache.httpcomponents#httpclient;4.5.9 in central
    found org.apache.httpcomponents#httpcore;4.4.11 in central
    found commons-codec#commons-codec;1.11 in central
    found software.amazon.ion#ion-java;1.0.2 in central
    found com.fasterxml.jackson.dataformat#jackson-dataformat-cbor;2.6.7 in central
    found com.amazonaws#aws-java-sdk-s3;1.11.603 in central
    found com.amazonaws#aws-java-sdk-kms;1.11.603 in central
    found com.amazonaws#jmespath-java;1.11.603 in central
    found com.fasterxml.jackson.core#jackson-databind;2.6.7.2 in central
    found com.fasterxml.jackson.core#jackson-annotations;2.6.0 in central
    found com.fasterxml.jackson.core#jackson-core;2.6.7 in central
    found com.github.universal-automata#liblevenshtein;3.0.0 in central
    found com.google.code.findbugs#annotations;3.0.1 in central
    found net.jcip#jcip-annotations;1.0 in central
    found com.google.code.findbugs#jsr305;3.0.1 in central
    found com.google.protobuf#protobuf-java-util;3.0.0-beta-3 in central
    found com.google.protobuf#protobuf-java;3.0.0-beta-3 in central
    found com.google.code.gson#gson;2.3 in central
    found it.unimi.dsi#fastutil;7.0.12 in central
    found org.projectlombok#lombok;1.16.8 in central
    found org.slf4j#slf4j-api;1.7.21 in central
    found com.navigamez#greex;1.0 in central
    found dk.brics.automaton#automaton;1.11-8 in central
    found org.json4s#json4s-ext_2.11;3.5.3 in central
    found joda-time#joda-time;2.9.5 in central
    found org.joda#joda-convert;1.8.1 in central
    found org.tensorflow#tensorflow;1.15.0 in central
    found org.tensorflow#libtensorflow;1.15.0 in central
    found org.tensorflow#libtensorflow_jni;1.15.0 in central
    found net.sf.trove4j#trove4j;3.0.3 in central
downloading https://repo1.maven.org/maven2/com/johnsnowlabs/nlp/spark-nlp_2.11/2.5.4/spark-nlp_2.11-2.5.4.jar ...
    [SUCCESSFUL ] com.johnsnowlabs.nlp#spark-nlp_2.11;2.5.4!spark-nlp_2.11.jar (1164ms)
downloading https://repo1.maven.org/maven2/com/typesafe/config/1.3.0/config-1.3.0.jar ...
    [SUCCESSFUL ] com.typesafe#config;1.3.0!config.jar(bundle) (25ms)
downloading https://repo1.maven.org/maven2/org/rocksdb/rocksdbjni/6.5.3/rocksdbjni-6.5.3.jar ...
    [SUCCESSFUL ] org.rocksdb#rocksdbjni;6.5.3!rocksdbjni.jar (749ms)
downloading https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-aws/3.2.0/hadoop-aws-3.2.0.jar ...
    [SUCCESSFUL ] org.apache.hadoop#hadoop-aws;3.2.0!hadoop-aws.jar (27ms)
downloading https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.603/aws-java-sdk-core-1.11.603.jar ...
    [SUCCESSFUL ] com.amazonaws#aws-java-sdk-core;1.11.603!aws-java-sdk-core.jar (41ms)
downloading https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.603/aws-java-sdk-s3-1.11.603.jar ...
    [SUCCESSFUL ] com.amazonaws#aws-java-sdk-s3;1.11.603!aws-java-sdk-s3.jar (42ms)
downloading https://repo1.maven.org/maven2/com/github/universal-automata/liblevenshtein/3.0.0/liblevenshtein-3.0.0.jar ...
    [SUCCESSFUL ] com.github.universal-automata#liblevenshtein;3.0.0!liblevenshtein.jar (20ms)
downloading https://repo1.maven.org/maven2/com/navigamez/greex/1.0/greex-1.0.jar ...
    [SUCCESSFUL ] com.navigamez#greex;1.0!greex.jar (16ms)
downloading https://repo1.maven.org/maven2/org/json4s/json4s-ext_2.11/3.5.3/json4s-ext_2.11-3.5.3.jar ...
    [SUCCESSFUL ] org.json4s#json4s-ext_2.11;3.5.3!json4s-ext_2.11.jar (20ms)
downloading https://repo1.maven.org/maven2/org/tensorflow/tensorflow/1.15.0/tensorflow-1.15.0.jar ...
    [SUCCESSFUL ] org.tensorflow#tensorflow;1.15.0!tensorflow.jar (17ms)
downloading https://repo1.maven.org/maven2/net/sf/trove4j/trove4j/3.0.3/trove4j-3.0.3.jar ...
    [SUCCESSFUL ] net.sf.trove4j#trove4j;3.0.3!trove4j.jar (79ms)
downloading https://repo1.maven.org/maven2/org/apache/httpcomponents/httpclient/4.5.9/httpclient-4.5.9.jar ...
    [SUCCESSFUL ] org.apache.httpcomponents#httpclient;4.5.9!httpclient.jar (32ms)
downloading https://repo1.maven.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.jar ...
    [SUCCESSFUL ] software.amazon.ion#ion-java;1.0.2!ion-java.jar(bundle) (27ms)
downloading https://repo1.maven.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.jar ...
    [SUCCESSFUL ] com.fasterxml.jackson.dataformat#jackson-dataformat-cbor;2.6.7!jackson-dataformat-cbor.jar(bundle) (16ms)
downloading https://repo1.maven.org/maven2/org/apache/httpcomponents/httpcore/4.4.11/httpcore-4.4.11.jar ...
    [SUCCESSFUL ] org.apache.httpcomponents#httpcore;4.4.11!httpcore.jar (24ms)
downloading https://repo1.maven.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.jar ...
    [SUCCESSFUL ] commons-codec#commons-codec;1.11!commons-codec.jar (25ms)
downloading https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.603/aws-java-sdk-kms-1.11.603.jar ...
    [SUCCESSFUL ] com.amazonaws#aws-java-sdk-kms;1.11.603!aws-java-sdk-kms.jar (51ms)
downloading https://repo1.maven.org/maven2/com/amazonaws/jmespath-java/1.11.603/jmespath-java-1.11.603.jar ...
    [SUCCESSFUL ] com.amazonaws#jmespath-java;1.11.603!jmespath-java.jar (16ms)
downloading https://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.2/jackson-databind-2.6.7.2.jar ...
    [SUCCESSFUL ] com.fasterxml.jackson.core#jackson-databind;2.6.7.2!jackson-databind.jar(bundle) (72ms)
downloading https://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.jar ...
    [SUCCESSFUL ] com.fasterxml.jackson.core#jackson-annotations;2.6.0!jackson-annotations.jar(bundle) (16ms)
downloading https://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.jar ...
    [SUCCESSFUL ] com.fasterxml.jackson.core#jackson-core;2.6.7!jackson-core.jar(bundle) (21ms)
downloading https://repo1.maven.org/maven2/com/google/code/findbugs/annotations/3.0.1/annotations-3.0.1.jar ...
    [SUCCESSFUL ] com.google.code.findbugs#annotations;3.0.1!annotations.jar (16ms)
downloading https://repo1.maven.org/maven2/com/google/protobuf/protobuf-java-util/3.0.0-beta-3/protobuf-java-util-3.0.0-beta-3.jar ...
    [SUCCESSFUL ] com.google.protobuf#protobuf-java-util;3.0.0-beta-3!protobuf-java-util.jar(bundle) (17ms)
downloading https://repo1.maven.org/maven2/com/google/protobuf/protobuf-java/3.0.0-beta-3/protobuf-java-3.0.0-beta-3.jar ...
    [SUCCESSFUL ] com.google.protobuf#protobuf-java;3.0.0-beta-3!protobuf-java.jar(bundle) (47ms)
downloading https://repo1.maven.org/maven2/it/unimi/dsi/fastutil/7.0.12/fastutil-7.0.12.jar ...
    [SUCCESSFUL ] it.unimi.dsi#fastutil;7.0.12!fastutil.jar (503ms)
downloading https://repo1.maven.org/maven2/org/projectlombok/lombok/1.16.8/lombok-1.16.8.jar ...
    [SUCCESSFUL ] org.projectlombok#lombok;1.16.8!lombok.jar (74ms)
downloading https://repo1.maven.org/maven2/org/slf4j/slf4j-api/1.7.21/slf4j-api-1.7.21.jar ...
    [SUCCESSFUL ] org.slf4j#slf4j-api;1.7.21!slf4j-api.jar (17ms)
downloading https://repo1.maven.org/maven2/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.jar ...
    [SUCCESSFUL ] net.jcip#jcip-annotations;1.0!jcip-annotations.jar (16ms)
downloading https://repo1.maven.org/maven2/com/google/code/findbugs/jsr305/3.0.1/jsr305-3.0.1.jar ...
    [SUCCESSFUL ] com.google.code.findbugs#jsr305;3.0.1!jsr305.jar (16ms)
downloading https://repo1.maven.org/maven2/com/google/code/gson/gson/2.3/gson-2.3.jar ...
    [SUCCESSFUL ] com.google.code.gson#gson;2.3!gson.jar (23ms)
downloading https://repo1.maven.org/maven2/dk/brics/automaton/automaton/1.11-8/automaton-1.11-8.jar ...
    [SUCCESSFUL ] dk.brics.automaton#automaton;1.11-8!automaton.jar (24ms)
downloading https://repo1.maven.org/maven2/joda-time/joda-time/2.9.5/joda-time-2.9.5.jar ...
    [SUCCESSFUL ] joda-time#joda-time;2.9.5!joda-time.jar (45ms)
downloading https://repo1.maven.org/maven2/org/joda/joda-convert/1.8.1/joda-convert-1.8.1.jar ...
    [SUCCESSFUL ] org.joda#joda-convert;1.8.1!joda-convert.jar (21ms)
downloading https://repo1.maven.org/maven2/org/tensorflow/libtensorflow/1.15.0/libtensorflow-1.15.0.jar ...
    [SUCCESSFUL ] org.tensorflow#libtensorflow;1.15.0!libtensorflow.jar (105ms)
downloading https://repo1.maven.org/maven2/org/tensorflow/libtensorflow_jni/1.15.0/libtensorflow_jni-1.15.0.jar ...
    [SUCCESSFUL ] org.tensorflow#libtensorflow_jni;1.15.0!libtensorflow_jni.jar (6378ms)
:: resolution report :: resolve 21975ms :: artifacts dl 9830ms
    :: modules in use:
    antlr#antlr;2.7.7 from central in [default]
    com.amazonaws#aws-java-sdk-core;1.11.603 from central in [default]
    com.amazonaws#aws-java-sdk-kms;1.11.603 from central in [default]
    com.amazonaws#aws-java-sdk-s3;1.11.603 from central in [default]
    com.amazonaws#jmespath-java;1.11.603 from central in [default]
    com.fasterxml.jackson.core#jackson-annotations;2.6.0 from central in [default]
    com.fasterxml.jackson.core#jackson-core;2.6.7 from central in [default]
    com.fasterxml.jackson.core#jackson-databind;2.6.7.2 from central in [default]
    com.fasterxml.jackson.dataformat#jackson-dataformat-cbor;2.6.7 from central in [default]
    com.github.universal-automata#liblevenshtein;3.0.0 from central in [default]
    com.google.code.findbugs#annotations;3.0.1 from central in [default]
    com.google.code.findbugs#jsr305;3.0.1 from central in [default]
    com.google.code.gson#gson;2.3 from central in [default]
    com.google.protobuf#protobuf-java;3.0.0-beta-3 from central in [default]
    com.google.protobuf#protobuf-java-util;3.0.0-beta-3 from central in [default]
    com.googlecode.javaewah#JavaEWAH;0.3.2 from central in [default]
    com.johnsnowlabs.nlp#spark-nlp_2.11;2.5.4 from central in [default]
    com.jolbox#bonecp;0.8.0.RELEASE from central in [default]
    com.navigamez#greex;1.0 from central in [default]
    com.thoughtworks.paranamer#paranamer;2.8 from central in [default]
    com.twitter#parquet-hadoop-bundle;1.6.0 from central in [default]
    com.typesafe#config;1.3.0 from central in [default]
    commons-cli#commons-cli;1.2 from central in [default]
    commons-codec#commons-codec;1.11 from central in [default]
    commons-dbcp#commons-dbcp;1.4 from central in [default]
    commons-httpclient#commons-httpclient;3.1 from central in [default]
    commons-io#commons-io;2.4 from central in [default]
    commons-lang#commons-lang;2.6 from central in [default]
    commons-logging#commons-logging;1.2 from central in [default]
    commons-pool#commons-pool;1.5.4 from central in [default]
    dk.brics.automaton#automaton;1.11-8 from central in [default]
    it.unimi.dsi#fastutil;7.0.12 from central in [default]
    javax.jdo#jdo-api;3.0.1 from central in [default]
    javax.transaction#jta;1.1 from central in [default]
    javolution#javolution;5.5.1 from central in [default]
    joda-time#joda-time;2.9.5 from central in [default]
    log4j#apache-log4j-extras;1.2.17 from central in [default]
    net.hydromatic#eigenbase-properties;1.1.5 from central in [default]
    net.jcip#jcip-annotations;1.0 from central in [default]
    net.sf.opencsv#opencsv;2.3 from central in [default]
    net.sf.trove4j#trove4j;3.0.3 from central in [default]
    org.antlr#ST4;4.0.4 from central in [default]
    org.antlr#antlr-runtime;3.4 from central in [default]
    org.antlr#stringtemplate;3.2.1 from central in [default]
    org.apache.avro#avro;1.8.2 from central in [default]
    org.apache.avro#avro-ipc;1.8.2 from central in [default]
    org.apache.avro#avro-mapred;1.8.2 from central in [default]
    org.apache.calcite#calcite-avatica;1.2.0-incubating from central in [default]
    org.apache.calcite#calcite-core;1.2.0-incubating from central in [default]
    org.apache.calcite#calcite-linq4j;1.2.0-incubating from central in [default]
    org.apache.commons#commons-compress;1.4.1 from central in [default]
    org.apache.commons#commons-lang3;3.5 from central in [default]
    org.apache.derby#derby;10.12.1.1 from central in [default]
    org.apache.hadoop#hadoop-aws;3.2.0 from central in [default]
    org.apache.httpcomponents#httpclient;4.5.9 from central in [default]
    org.apache.httpcomponents#httpcore;4.4.11 from central in [default]
    org.apache.ivy#ivy;2.4.0 from central in [default]
    org.apache.spark#spark-hive_2.11;2.4.0 from central in [default]
    org.apache.thrift#libfb303;0.9.3 from central in [default]
    org.apache.thrift#libthrift;0.9.3 from central in [default]
    org.codehaus.jackson#jackson-core-asl;1.9.13 from central in [default]
    org.codehaus.jackson#jackson-mapper-asl;1.9.13 from central in [default]
    org.datanucleus#datanucleus-api-jdo;3.2.6 from central in [default]
    org.datanucleus#datanucleus-core;3.2.10 from central in [default]
    org.datanucleus#datanucleus-rdbms;3.2.9 from central in [default]
    org.iq80.snappy#snappy;0.2 from central in [default]
    org.joda#joda-convert;1.8.1 from central in [default]
    org.jodd#jodd-core;3.5.2 from central in [default]
    org.json4s#json4s-ext_2.11;3.5.3 from central in [default]
    org.projectlombok#lombok;1.16.8 from central in [default]
    org.rocksdb#rocksdbjni;6.5.3 from central in [default]
    org.slf4j#slf4j-api;1.7.21 from central in [default]
    org.spark-project.hive#hive-exec;1.2.1.spark2 from central in [default]
    org.spark-project.hive#hive-metastore;1.2.1.spark2 from central in [default]
    org.spark-project.spark#unused;1.0.0 from central in [default]
    org.tensorflow#libtensorflow;1.15.0 from central in [default]
    org.tensorflow#libtensorflow_jni;1.15.0 from central in [default]
    org.tensorflow#tensorflow;1.15.0 from central in [default]
    org.tukaani#xz;1.5 from central in [default]
    org.xerial.snappy#snappy-java;1.1.7.1 from central in [default]
    software.amazon.ion#ion-java;1.0.2 from central in [default]
    stax#stax-api;1.0.1 from central in [default]
    :: evicted modules:
    org.tukaani#xz;1.0 by [org.tukaani#xz;1.5] in [default]
    org.slf4j#slf4j-api;1.7.16 by [org.slf4j#slf4j-api;1.7.21] in [default]
    commons-logging#commons-logging;1.1.3 by [commons-logging#commons-logging;1.2] in [default]
    commons-codec#commons-codec;1.10 by [commons-codec#commons-codec;1.11] in [default]
    commons-logging#commons-logging;1.0.4 by [commons-logging#commons-logging;1.1.3] in [default]
    org.apache.httpcomponents#httpclient;4.5.6 by [org.apache.httpcomponents#httpclient;4.5.9] in [default]
    org.apache.httpcomponents#httpcore;4.4.10 by [org.apache.httpcomponents#httpcore;4.4.11] in [default]
    joda-time#joda-time;2.9.3 by [joda-time#joda-time;2.9.5] in [default]
    com.google.code.findbugs#jsr305;1.3.9 by [com.google.code.findbugs#jsr305;3.0.1] in [default]
    joda-time#joda-time;2.8.1 by [joda-time#joda-time;2.9.5] in [default]
    ---------------------------------------------------------------------
    |                  |            modules            ||   artifacts   |
    |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
    ---------------------------------------------------------------------
    |      default     |   92  |   35  |   35  |   10  ||   82  |   35  |
    ---------------------------------------------------------------------
:: retrieving :: org.apache.spark#spark-submit-parent-d3ee78bf-b90d-4418-bbd7-02772086063f
    confs: [default]
    35 artifacts copied, 47 already retrieved (226314kB/960ms)
2020-08-06 11:34:12 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2020-08-06 11:34:12 WARN  DependencyUtils:66 - Local jar /root/.ivy2/jars/org.apache.avro_avro-mapred-1.8.2.jar does not exist, skipping.
2020-08-06 11:34:12 INFO  SparkContext:54 - Running Spark version 2.4.0
2020-08-06 11:34:12 INFO  SparkContext:54 - Submitted application: spark-nlp-starter
2020-08-06 11:34:12 INFO  SecurityManager:54 - Changing view acls to: root
2020-08-06 11:34:12 INFO  SecurityManager:54 - Changing modify acls to: root
2020-08-06 11:34:12 INFO  SecurityManager:54 - Changing view acls groups to: 
2020-08-06 11:34:12 INFO  SecurityManager:54 - Changing modify acls groups to: 
2020-08-06 11:34:12 INFO  SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
2020-08-06 11:34:13 INFO  Utils:54 - Successfully started service 'sparkDriver' on port 32961.
2020-08-06 11:34:13 INFO  SparkEnv:54 - Registering MapOutputTracker
2020-08-06 11:34:13 INFO  SparkEnv:54 - Registering BlockManagerMaster
2020-08-06 11:34:13 INFO  BlockManagerMasterEndpoint:54 - Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
2020-08-06 11:34:13 INFO  BlockManagerMasterEndpoint:54 - BlockManagerMasterEndpoint up
2020-08-06 11:34:13 INFO  DiskBlockManager:54 - Created local directory at /tmp/blockmgr-d5e457bc-d7ee-4fe7-adbd-1cf45693a7d9
2020-08-06 11:34:13 INFO  MemoryStore:54 - MemoryStore started with capacity 2004.6 MB
2020-08-06 11:34:13 INFO  SparkEnv:54 - Registering OutputCommitCoordinator
2020-08-06 11:34:13 INFO  log:192 - Logging initialized @35514ms
2020-08-06 11:34:13 INFO  Server:351 - jetty-9.3.z-SNAPSHOT, build timestamp: unknown, git hash: unknown
2020-08-06 11:34:13 INFO  Server:419 - Started @35617ms
2020-08-06 11:34:13 INFO  AbstractConnector:278 - Started ServerConnector@10c72a6f{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
2020-08-06 11:34:13 INFO  Utils:54 - Successfully started service 'SparkUI' on port 4040.
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@6ae62c7e{/jobs,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@48a0c8aa{/jobs/json,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@6192a5d5{/jobs/job,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@4cbc2e3b{/jobs/job/json,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@2975a9e{/stages,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@765ffb14{/stages/json,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@57562473{/stages/stage,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@4bc33720{/stages/stage/json,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@2dd0f797{/stages/pool,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@67064bdc{/stages/pool/json,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@4a7fd0c9{/storage,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@18578491{/storage/json,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@3291b443{/storage/rdd,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@671c4166{/storage/rdd/json,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@53d13cd4{/environment,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@77865933{/environment/json,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@480ad82c{/executors,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@4d18b73a{/executors/json,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@177515d1{/executors/threadDump,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@52ff99cd{/executors/threadDump/json,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@4c2af006{/static,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@77049094{/,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@f88bfbe{/api,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@446626a7{/jobs/job/kill,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@429f7919{/stages/stage/kill,null,AVAILABLE,@Spark}
2020-08-06 11:34:13 INFO  SparkUI:54 - Bound SparkUI to 0.0.0.0, and started at http://be6d9adbe68c:4040
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.apache.spark_spark-hive_2.11-2.4.0.jar at spark://be6d9adbe68c:32961/jars/org.apache.spark_spark-hive_2.11-2.4.0.jar with timestamp 1596713653958
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.johnsnowlabs.nlp_spark-nlp_2.11-2.5.4.jar at spark://be6d9adbe68c:32961/jars/com.johnsnowlabs.nlp_spark-nlp_2.11-2.5.4.jar with timestamp 1596713653959
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.twitter_parquet-hadoop-bundle-1.6.0.jar at spark://be6d9adbe68c:32961/jars/com.twitter_parquet-hadoop-bundle-1.6.0.jar with timestamp 1596713653959
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.spark-project.hive_hive-exec-1.2.1.spark2.jar at spark://be6d9adbe68c:32961/jars/org.spark-project.hive_hive-exec-1.2.1.spark2.jar with timestamp 1596713653959
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.spark-project.hive_hive-metastore-1.2.1.spark2.jar at spark://be6d9adbe68c:32961/jars/org.spark-project.hive_hive-metastore-1.2.1.spark2.jar with timestamp 1596713653960
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.apache.avro_avro-1.8.2.jar at spark://be6d9adbe68c:32961/jars/org.apache.avro_avro-1.8.2.jar with timestamp 1596713653960
2020-08-06 11:34:13 ERROR SparkContext:91 - Failed to add file:/root/.ivy2/jars/org.apache.avro_avro-mapred-1.8.2.jar to Spark environment
java.io.FileNotFoundException: Jar /root/.ivy2/jars/org.apache.avro_avro-mapred-1.8.2.jar not found
    at org.apache.spark.SparkContext.addJarFile$1(SparkContext.scala:1838)
    at org.apache.spark.SparkContext.addJar(SparkContext.scala:1868)
    at org.apache.spark.SparkContext$$anonfun$12.apply(SparkContext.scala:458)
    at org.apache.spark.SparkContext$$anonfun$12.apply(SparkContext.scala:458)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:458)
    at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2520)
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:935)
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:926)
    at scala.Option.getOrElse(Option.scala:121)
    at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:926)
    at Main$.main(Main.scala:13)
    at Main.main(Main.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
    at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
    at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
    at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
    at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
    at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/commons-httpclient_commons-httpclient-3.1.jar at spark://be6d9adbe68c:32961/jars/commons-httpclient_commons-httpclient-3.1.jar with timestamp 1596713653964
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.apache.calcite_calcite-avatica-1.2.0-incubating.jar at spark://be6d9adbe68c:32961/jars/org.apache.calcite_calcite-avatica-1.2.0-incubating.jar with timestamp 1596713653964
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.apache.calcite_calcite-core-1.2.0-incubating.jar at spark://be6d9adbe68c:32961/jars/org.apache.calcite_calcite-core-1.2.0-incubating.jar with timestamp 1596713653965
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.codehaus.jackson_jackson-mapper-asl-1.9.13.jar at spark://be6d9adbe68c:32961/jars/org.codehaus.jackson_jackson-mapper-asl-1.9.13.jar with timestamp 1596713653965
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.jodd_jodd-core-3.5.2.jar at spark://be6d9adbe68c:32961/jars/org.jodd_jodd-core-3.5.2.jar with timestamp 1596713653965
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.datanucleus_datanucleus-core-3.2.10.jar at spark://be6d9adbe68c:32961/jars/org.datanucleus_datanucleus-core-3.2.10.jar with timestamp 1596713653966
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.apache.thrift_libthrift-0.9.3.jar at spark://be6d9adbe68c:32961/jars/org.apache.thrift_libthrift-0.9.3.jar with timestamp 1596713653966
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.apache.thrift_libfb303-0.9.3.jar at spark://be6d9adbe68c:32961/jars/org.apache.thrift_libfb303-0.9.3.jar with timestamp 1596713653966
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.apache.derby_derby-10.12.1.1.jar at spark://be6d9adbe68c:32961/jars/org.apache.derby_derby-10.12.1.1.jar with timestamp 1596713653966
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.spark-project.spark_unused-1.0.0.jar at spark://be6d9adbe68c:32961/jars/org.spark-project.spark_unused-1.0.0.jar with timestamp 1596713653967
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/commons-io_commons-io-2.4.jar at spark://be6d9adbe68c:32961/jars/commons-io_commons-io-2.4.jar with timestamp 1596713653967
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.apache.commons_commons-lang3-3.5.jar at spark://be6d9adbe68c:32961/jars/org.apache.commons_commons-lang3-3.5.jar with timestamp 1596713653967
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/commons-lang_commons-lang-2.6.jar at spark://be6d9adbe68c:32961/jars/commons-lang_commons-lang-2.6.jar with timestamp 1596713653968
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/javolution_javolution-5.5.1.jar at spark://be6d9adbe68c:32961/jars/javolution_javolution-5.5.1.jar with timestamp 1596713653968
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/log4j_apache-log4j-extras-1.2.17.jar at spark://be6d9adbe68c:32961/jars/log4j_apache-log4j-extras-1.2.17.jar with timestamp 1596713653968
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.antlr_antlr-runtime-3.4.jar at spark://be6d9adbe68c:32961/jars/org.antlr_antlr-runtime-3.4.jar with timestamp 1596713653969
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.antlr_ST4-4.0.4.jar at spark://be6d9adbe68c:32961/jars/org.antlr_ST4-4.0.4.jar with timestamp 1596713653969
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.apache.commons_commons-compress-1.4.1.jar at spark://be6d9adbe68c:32961/jars/org.apache.commons_commons-compress-1.4.1.jar with timestamp 1596713653969
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.apache.ivy_ivy-2.4.0.jar at spark://be6d9adbe68c:32961/jars/org.apache.ivy_ivy-2.4.0.jar with timestamp 1596713653970
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.codehaus.jackson_jackson-core-asl-1.9.13.jar at spark://be6d9adbe68c:32961/jars/org.codehaus.jackson_jackson-core-asl-1.9.13.jar with timestamp 1596713653970
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.googlecode.javaewah_JavaEWAH-0.3.2.jar at spark://be6d9adbe68c:32961/jars/com.googlecode.javaewah_JavaEWAH-0.3.2.jar with timestamp 1596713653970
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.iq80.snappy_snappy-0.2.jar at spark://be6d9adbe68c:32961/jars/org.iq80.snappy_snappy-0.2.jar with timestamp 1596713653970
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/stax_stax-api-1.0.1.jar at spark://be6d9adbe68c:32961/jars/stax_stax-api-1.0.1.jar with timestamp 1596713653971
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/net.sf.opencsv_opencsv-2.3.jar at spark://be6d9adbe68c:32961/jars/net.sf.opencsv_opencsv-2.3.jar with timestamp 1596713653971
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.antlr_stringtemplate-3.2.1.jar at spark://be6d9adbe68c:32961/jars/org.antlr_stringtemplate-3.2.1.jar with timestamp 1596713653971
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/antlr_antlr-2.7.7.jar at spark://be6d9adbe68c:32961/jars/antlr_antlr-2.7.7.jar with timestamp 1596713653972
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.thoughtworks.paranamer_paranamer-2.8.jar at spark://be6d9adbe68c:32961/jars/com.thoughtworks.paranamer_paranamer-2.8.jar with timestamp 1596713653972
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.xerial.snappy_snappy-java-1.1.7.1.jar at spark://be6d9adbe68c:32961/jars/org.xerial.snappy_snappy-java-1.1.7.1.jar with timestamp 1596713653973
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.tukaani_xz-1.5.jar at spark://be6d9adbe68c:32961/jars/org.tukaani_xz-1.5.jar with timestamp 1596713653973
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.jolbox_bonecp-0.8.0.RELEASE.jar at spark://be6d9adbe68c:32961/jars/com.jolbox_bonecp-0.8.0.RELEASE.jar with timestamp 1596713653974
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/commons-cli_commons-cli-1.2.jar at spark://be6d9adbe68c:32961/jars/commons-cli_commons-cli-1.2.jar with timestamp 1596713653974
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.datanucleus_datanucleus-api-jdo-3.2.6.jar at spark://be6d9adbe68c:32961/jars/org.datanucleus_datanucleus-api-jdo-3.2.6.jar with timestamp 1596713653975
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.datanucleus_datanucleus-rdbms-3.2.9.jar at spark://be6d9adbe68c:32961/jars/org.datanucleus_datanucleus-rdbms-3.2.9.jar with timestamp 1596713653975
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/commons-pool_commons-pool-1.5.4.jar at spark://be6d9adbe68c:32961/jars/commons-pool_commons-pool-1.5.4.jar with timestamp 1596713653975
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/commons-dbcp_commons-dbcp-1.4.jar at spark://be6d9adbe68c:32961/jars/commons-dbcp_commons-dbcp-1.4.jar with timestamp 1596713653976
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/javax.jdo_jdo-api-3.0.1.jar at spark://be6d9adbe68c:32961/jars/javax.jdo_jdo-api-3.0.1.jar with timestamp 1596713653977
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/javax.transaction_jta-1.1.jar at spark://be6d9adbe68c:32961/jars/javax.transaction_jta-1.1.jar with timestamp 1596713653977
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.apache.avro_avro-ipc-1.8.2.jar at spark://be6d9adbe68c:32961/jars/org.apache.avro_avro-ipc-1.8.2.jar with timestamp 1596713653978
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.apache.calcite_calcite-linq4j-1.2.0-incubating.jar at spark://be6d9adbe68c:32961/jars/org.apache.calcite_calcite-linq4j-1.2.0-incubating.jar with timestamp 1596713653978
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/net.hydromatic_eigenbase-properties-1.1.5.jar at spark://be6d9adbe68c:32961/jars/net.hydromatic_eigenbase-properties-1.1.5.jar with timestamp 1596713653979
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/commons-logging_commons-logging-1.2.jar at spark://be6d9adbe68c:32961/jars/commons-logging_commons-logging-1.2.jar with timestamp 1596713653979
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.typesafe_config-1.3.0.jar at spark://be6d9adbe68c:32961/jars/com.typesafe_config-1.3.0.jar with timestamp 1596713653980
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.rocksdb_rocksdbjni-6.5.3.jar at spark://be6d9adbe68c:32961/jars/org.rocksdb_rocksdbjni-6.5.3.jar with timestamp 1596713653980
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.apache.hadoop_hadoop-aws-3.2.0.jar at spark://be6d9adbe68c:32961/jars/org.apache.hadoop_hadoop-aws-3.2.0.jar with timestamp 1596713653985
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.amazonaws_aws-java-sdk-core-1.11.603.jar at spark://be6d9adbe68c:32961/jars/com.amazonaws_aws-java-sdk-core-1.11.603.jar with timestamp 1596713653988
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.amazonaws_aws-java-sdk-s3-1.11.603.jar at spark://be6d9adbe68c:32961/jars/com.amazonaws_aws-java-sdk-s3-1.11.603.jar with timestamp 1596713653988
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.github.universal-automata_liblevenshtein-3.0.0.jar at spark://be6d9adbe68c:32961/jars/com.github.universal-automata_liblevenshtein-3.0.0.jar with timestamp 1596713653988
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.navigamez_greex-1.0.jar at spark://be6d9adbe68c:32961/jars/com.navigamez_greex-1.0.jar with timestamp 1596713653989
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.json4s_json4s-ext_2.11-3.5.3.jar at spark://be6d9adbe68c:32961/jars/org.json4s_json4s-ext_2.11-3.5.3.jar with timestamp 1596713653989
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.tensorflow_tensorflow-1.15.0.jar at spark://be6d9adbe68c:32961/jars/org.tensorflow_tensorflow-1.15.0.jar with timestamp 1596713653989
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/net.sf.trove4j_trove4j-3.0.3.jar at spark://be6d9adbe68c:32961/jars/net.sf.trove4j_trove4j-3.0.3.jar with timestamp 1596713653989
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.apache.httpcomponents_httpclient-4.5.9.jar at spark://be6d9adbe68c:32961/jars/org.apache.httpcomponents_httpclient-4.5.9.jar with timestamp 1596713653990
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/software.amazon.ion_ion-java-1.0.2.jar at spark://be6d9adbe68c:32961/jars/software.amazon.ion_ion-java-1.0.2.jar with timestamp 1596713653990
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.fasterxml.jackson.dataformat_jackson-dataformat-cbor-2.6.7.jar at spark://be6d9adbe68c:32961/jars/com.fasterxml.jackson.dataformat_jackson-dataformat-cbor-2.6.7.jar with timestamp 1596713653990
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.apache.httpcomponents_httpcore-4.4.11.jar at spark://be6d9adbe68c:32961/jars/org.apache.httpcomponents_httpcore-4.4.11.jar with timestamp 1596713653991
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/commons-codec_commons-codec-1.11.jar at spark://be6d9adbe68c:32961/jars/commons-codec_commons-codec-1.11.jar with timestamp 1596713653991
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.amazonaws_aws-java-sdk-kms-1.11.603.jar at spark://be6d9adbe68c:32961/jars/com.amazonaws_aws-java-sdk-kms-1.11.603.jar with timestamp 1596713653992
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.amazonaws_jmespath-java-1.11.603.jar at spark://be6d9adbe68c:32961/jars/com.amazonaws_jmespath-java-1.11.603.jar with timestamp 1596713653992
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.fasterxml.jackson.core_jackson-databind-2.6.7.2.jar at spark://be6d9adbe68c:32961/jars/com.fasterxml.jackson.core_jackson-databind-2.6.7.2.jar with timestamp 1596713653993
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.fasterxml.jackson.core_jackson-annotations-2.6.0.jar at spark://be6d9adbe68c:32961/jars/com.fasterxml.jackson.core_jackson-annotations-2.6.0.jar with timestamp 1596713653993
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.fasterxml.jackson.core_jackson-core-2.6.7.jar at spark://be6d9adbe68c:32961/jars/com.fasterxml.jackson.core_jackson-core-2.6.7.jar with timestamp 1596713653994
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.google.code.findbugs_annotations-3.0.1.jar at spark://be6d9adbe68c:32961/jars/com.google.code.findbugs_annotations-3.0.1.jar with timestamp 1596713653994
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.google.protobuf_protobuf-java-util-3.0.0-beta-3.jar at spark://be6d9adbe68c:32961/jars/com.google.protobuf_protobuf-java-util-3.0.0-beta-3.jar with timestamp 1596713653994
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.google.protobuf_protobuf-java-3.0.0-beta-3.jar at spark://be6d9adbe68c:32961/jars/com.google.protobuf_protobuf-java-3.0.0-beta-3.jar with timestamp 1596713653995
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/it.unimi.dsi_fastutil-7.0.12.jar at spark://be6d9adbe68c:32961/jars/it.unimi.dsi_fastutil-7.0.12.jar with timestamp 1596713653995
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.projectlombok_lombok-1.16.8.jar at spark://be6d9adbe68c:32961/jars/org.projectlombok_lombok-1.16.8.jar with timestamp 1596713653995
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.slf4j_slf4j-api-1.7.21.jar at spark://be6d9adbe68c:32961/jars/org.slf4j_slf4j-api-1.7.21.jar with timestamp 1596713653996
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/net.jcip_jcip-annotations-1.0.jar at spark://be6d9adbe68c:32961/jars/net.jcip_jcip-annotations-1.0.jar with timestamp 1596713653996
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.google.code.findbugs_jsr305-3.0.1.jar at spark://be6d9adbe68c:32961/jars/com.google.code.findbugs_jsr305-3.0.1.jar with timestamp 1596713653997
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/com.google.code.gson_gson-2.3.jar at spark://be6d9adbe68c:32961/jars/com.google.code.gson_gson-2.3.jar with timestamp 1596713653997
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/dk.brics.automaton_automaton-1.11-8.jar at spark://be6d9adbe68c:32961/jars/dk.brics.automaton_automaton-1.11-8.jar with timestamp 1596713653997
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/joda-time_joda-time-2.9.5.jar at spark://be6d9adbe68c:32961/jars/joda-time_joda-time-2.9.5.jar with timestamp 1596713653998
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.joda_joda-convert-1.8.1.jar at spark://be6d9adbe68c:32961/jars/org.joda_joda-convert-1.8.1.jar with timestamp 1596713653998
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.tensorflow_libtensorflow-1.15.0.jar at spark://be6d9adbe68c:32961/jars/org.tensorflow_libtensorflow-1.15.0.jar with timestamp 1596713653999
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:///root/.ivy2/jars/org.tensorflow_libtensorflow_jni-1.15.0.jar at spark://be6d9adbe68c:32961/jars/org.tensorflow_libtensorflow_jni-1.15.0.jar with timestamp 1596713653999
2020-08-06 11:34:13 INFO  SparkContext:54 - Added JAR file:/deploy/spark-nlp-starter-fix.jar at spark://be6d9adbe68c:32961/jars/spark-nlp-starter-fix.jar with timestamp 1596713653999
2020-08-06 11:34:14 INFO  Executor:54 - Starting executor ID driver on host localhost
2020-08-06 11:34:14 INFO  Utils:54 - Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 36193.
2020-08-06 11:34:14 INFO  NettyBlockTransferService:54 - Server created on be6d9adbe68c:36193
2020-08-06 11:34:14 INFO  BlockManager:54 - Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
2020-08-06 11:34:14 INFO  BlockManagerMaster:54 - Registering BlockManager BlockManagerId(driver, be6d9adbe68c, 36193, None)
2020-08-06 11:34:14 INFO  BlockManagerMasterEndpoint:54 - Registering block manager be6d9adbe68c:36193 with 2004.6 MB RAM, BlockManagerId(driver, be6d9adbe68c, 36193, None)
2020-08-06 11:34:14 INFO  BlockManagerMaster:54 - Registered BlockManager BlockManagerId(driver, be6d9adbe68c, 36193, None)
2020-08-06 11:34:14 INFO  BlockManager:54 - Initialized BlockManager: BlockManagerId(driver, be6d9adbe68c, 36193, None)
2020-08-06 11:34:14 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@4b960b5b{/metrics/json,null,AVAILABLE,@Spark}
glove_100d download started this may take some time.
2020-08-06 11:34:15 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
2020-08-06 11:34:15 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
2020-08-06 11:34:15 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
2020-08-06 11:34:15 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
2020-08-06 11:34:16 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
2020-08-06 11:34:16 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
2020-08-06 11:34:16 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
2020-08-06 11:34:16 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
Approximate size to download 145.3 MB
2020-08-06 11:34:16 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
2020-08-06 11:34:16 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
Download done! Loading the resource.
2020-08-06 11:34:38 INFO  MemoryStore:54 - Block broadcast_0 stored as values in memory (estimated size 236.7 KB, free 2004.4 MB)
2020-08-06 11:34:39 INFO  MemoryStore:54 - Block broadcast_0_piece0 stored as bytes in memory (estimated size 22.9 KB, free 2004.3 MB)
2020-08-06 11:34:39 INFO  BlockManagerInfo:54 - Added broadcast_0_piece0 in memory on be6d9adbe68c:36193 (size: 22.9 KB, free: 2004.6 MB)
2020-08-06 11:34:39 INFO  SparkContext:54 - Created broadcast 0 from textFile at ReadWrite.scala:615
2020-08-06 11:34:39 INFO  FileInputFormat:249 - Total input paths to process : 1
2020-08-06 11:34:39 INFO  SparkContext:54 - Starting job: first at ReadWrite.scala:615
2020-08-06 11:34:39 INFO  DAGScheduler:54 - Got job 0 (first at ReadWrite.scala:615) with 1 output partitions
2020-08-06 11:34:39 INFO  DAGScheduler:54 - Final stage: ResultStage 0 (first at ReadWrite.scala:615)
2020-08-06 11:34:39 INFO  DAGScheduler:54 - Parents of final stage: List()
2020-08-06 11:34:39 INFO  DAGScheduler:54 - Missing parents: List()
2020-08-06 11:34:39 INFO  DAGScheduler:54 - Submitting ResultStage 0 (file:/root/cache_pretrained/glove_100d_en_2.4.0_2.4_1579690104032/metadata MapPartitionsRDD[1] at textFile at ReadWrite.scala:615), which has no missing parents
2020-08-06 11:34:39 INFO  MemoryStore:54 - Block broadcast_1 stored as values in memory (estimated size 3.5 KB, free 2004.3 MB)
2020-08-06 11:34:39 INFO  MemoryStore:54 - Block broadcast_1_piece0 stored as bytes in memory (estimated size 2.1 KB, free 2004.3 MB)
2020-08-06 11:34:39 INFO  BlockManagerInfo:54 - Added broadcast_1_piece0 in memory on be6d9adbe68c:36193 (size: 2.1 KB, free: 2004.6 MB)
2020-08-06 11:34:39 INFO  SparkContext:54 - Created broadcast 1 from broadcast at DAGScheduler.scala:1161
2020-08-06 11:34:39 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 0 (file:/root/cache_pretrained/glove_100d_en_2.4.0_2.4_1579690104032/metadata MapPartitionsRDD[1] at textFile at ReadWrite.scala:615) (first 15 tasks are for partitions Vector(0))
2020-08-06 11:34:39 INFO  TaskSchedulerImpl:54 - Adding task set 0.0 with 1 tasks
2020-08-06 11:34:39 INFO  TaskSetManager:54 - Starting task 0.0 in stage 0.0 (TID 0, localhost, executor driver, partition 0, PROCESS_LOCAL, 7945 bytes)
2020-08-06 11:34:39 INFO  Executor:54 - Running task 0.0 in stage 0.0 (TID 0)
2020-08-06 11:34:39 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.avro_avro-ipc-1.8.2.jar with timestamp 1596713653978
2020-08-06 11:34:39 INFO  TransportClientFactory:267 - Successfully created connection to be6d9adbe68c/172.19.0.4:32961 after 33 ms (0 ms spent in bootstraps)
2020-08-06 11:34:39 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.avro_avro-ipc-1.8.2.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp6319282086322931084.tmp
2020-08-06 11:34:39 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.apache.avro_avro-ipc-1.8.2.jar to class loader
2020-08-06 11:34:39 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/javax.transaction_jta-1.1.jar with timestamp 1596713653977
2020-08-06 11:34:39 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/javax.transaction_jta-1.1.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp2533374421830586698.tmp
2020-08-06 11:34:39 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/javax.transaction_jta-1.1.jar to class loader
2020-08-06 11:34:39 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.antlr_ST4-4.0.4.jar with timestamp 1596713653969
2020-08-06 11:34:39 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.antlr_ST4-4.0.4.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp1657910354724731103.tmp
2020-08-06 11:34:39 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.antlr_ST4-4.0.4.jar to class loader
2020-08-06 11:34:39 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/commons-pool_commons-pool-1.5.4.jar with timestamp 1596713653975
2020-08-06 11:34:39 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/commons-pool_commons-pool-1.5.4.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp7013132363793537085.tmp
2020-08-06 11:34:39 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/commons-pool_commons-pool-1.5.4.jar to class loader
2020-08-06 11:34:39 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.rocksdb_rocksdbjni-6.5.3.jar with timestamp 1596713653980
2020-08-06 11:34:39 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.rocksdb_rocksdbjni-6.5.3.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp384764109952367151.tmp
2020-08-06 11:34:39 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.rocksdb_rocksdbjni-6.5.3.jar to class loader
2020-08-06 11:34:39 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.spark-project.spark_unused-1.0.0.jar with timestamp 1596713653967
2020-08-06 11:34:39 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.spark-project.spark_unused-1.0.0.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp2722639281430810217.tmp
2020-08-06 11:34:39 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.spark-project.spark_unused-1.0.0.jar to class loader
2020-08-06 11:34:39 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.httpcomponents_httpclient-4.5.9.jar with timestamp 1596713653990
2020-08-06 11:34:39 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.httpcomponents_httpclient-4.5.9.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp6925762211429788760.tmp
2020-08-06 11:34:39 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.apache.httpcomponents_httpclient-4.5.9.jar to class loader
2020-08-06 11:34:39 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/net.sf.opencsv_opencsv-2.3.jar with timestamp 1596713653971
2020-08-06 11:34:39 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/net.sf.opencsv_opencsv-2.3.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp834498538306856372.tmp
2020-08-06 11:34:39 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/net.sf.opencsv_opencsv-2.3.jar to class loader
2020-08-06 11:34:39 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/it.unimi.dsi_fastutil-7.0.12.jar with timestamp 1596713653995
2020-08-06 11:34:39 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/it.unimi.dsi_fastutil-7.0.12.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp6516757532141978496.tmp
2020-08-06 11:34:39 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/it.unimi.dsi_fastutil-7.0.12.jar to class loader
2020-08-06 11:34:39 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.projectlombok_lombok-1.16.8.jar with timestamp 1596713653995
2020-08-06 11:34:39 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.projectlombok_lombok-1.16.8.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp230928038361604063.tmp
2020-08-06 11:34:39 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.projectlombok_lombok-1.16.8.jar to class loader
2020-08-06 11:34:39 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.tensorflow_libtensorflow-1.15.0.jar with timestamp 1596713653999
2020-08-06 11:34:39 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.tensorflow_libtensorflow-1.15.0.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp4325337659562686362.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.tensorflow_libtensorflow-1.15.0.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/joda-time_joda-time-2.9.5.jar with timestamp 1596713653998
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/joda-time_joda-time-2.9.5.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp1141331496498919457.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/joda-time_joda-time-2.9.5.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.thrift_libthrift-0.9.3.jar with timestamp 1596713653966
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.thrift_libthrift-0.9.3.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp2256369741605287634.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.apache.thrift_libthrift-0.9.3.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/commons-logging_commons-logging-1.2.jar with timestamp 1596713653979
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/commons-logging_commons-logging-1.2.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp7353640326902499444.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/commons-logging_commons-logging-1.2.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.amazonaws_aws-java-sdk-s3-1.11.603.jar with timestamp 1596713653988
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.amazonaws_aws-java-sdk-s3-1.11.603.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp2421814755301765979.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.amazonaws_aws-java-sdk-s3-1.11.603.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.datanucleus_datanucleus-core-3.2.10.jar with timestamp 1596713653966
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.datanucleus_datanucleus-core-3.2.10.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp3206386278729712418.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.datanucleus_datanucleus-core-3.2.10.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.googlecode.javaewah_JavaEWAH-0.3.2.jar with timestamp 1596713653970
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.googlecode.javaewah_JavaEWAH-0.3.2.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp396835148499781940.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.googlecode.javaewah_JavaEWAH-0.3.2.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.xerial.snappy_snappy-java-1.1.7.1.jar with timestamp 1596713653973
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.xerial.snappy_snappy-java-1.1.7.1.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp7204002272398324722.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.xerial.snappy_snappy-java-1.1.7.1.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.google.code.gson_gson-2.3.jar with timestamp 1596713653997
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.google.code.gson_gson-2.3.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp5992516585796635936.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.google.code.gson_gson-2.3.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.fasterxml.jackson.dataformat_jackson-dataformat-cbor-2.6.7.jar with timestamp 1596713653990
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.fasterxml.jackson.dataformat_jackson-dataformat-cbor-2.6.7.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp8890659939145484411.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.fasterxml.jackson.dataformat_jackson-dataformat-cbor-2.6.7.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.johnsnowlabs.nlp_spark-nlp_2.11-2.5.4.jar with timestamp 1596713653959
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.johnsnowlabs.nlp_spark-nlp_2.11-2.5.4.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp4650862620964539320.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.johnsnowlabs.nlp_spark-nlp_2.11-2.5.4.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.amazonaws_aws-java-sdk-kms-1.11.603.jar with timestamp 1596713653992
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.amazonaws_aws-java-sdk-kms-1.11.603.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp5384465425549556665.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.amazonaws_aws-java-sdk-kms-1.11.603.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/commons-codec_commons-codec-1.11.jar with timestamp 1596713653991
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/commons-codec_commons-codec-1.11.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp3050260511996346300.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/commons-codec_commons-codec-1.11.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.antlr_stringtemplate-3.2.1.jar with timestamp 1596713653971
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.antlr_stringtemplate-3.2.1.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp2219924874207072095.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.antlr_stringtemplate-3.2.1.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.thoughtworks.paranamer_paranamer-2.8.jar with timestamp 1596713653972
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.thoughtworks.paranamer_paranamer-2.8.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp6041873873860351782.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.thoughtworks.paranamer_paranamer-2.8.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.jolbox_bonecp-0.8.0.RELEASE.jar with timestamp 1596713653974
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.jolbox_bonecp-0.8.0.RELEASE.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp3002135575086073724.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.jolbox_bonecp-0.8.0.RELEASE.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/commons-dbcp_commons-dbcp-1.4.jar with timestamp 1596713653976
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/commons-dbcp_commons-dbcp-1.4.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp8012973424461323665.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/commons-dbcp_commons-dbcp-1.4.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.codehaus.jackson_jackson-core-asl-1.9.13.jar with timestamp 1596713653970
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.codehaus.jackson_jackson-core-asl-1.9.13.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp3392548109864078530.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.codehaus.jackson_jackson-core-asl-1.9.13.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.calcite_calcite-avatica-1.2.0-incubating.jar with timestamp 1596713653964
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.calcite_calcite-avatica-1.2.0-incubating.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp91127179653184622.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.apache.calcite_calcite-avatica-1.2.0-incubating.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.ivy_ivy-2.4.0.jar with timestamp 1596713653970
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.ivy_ivy-2.4.0.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp7910908569915599228.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.apache.ivy_ivy-2.4.0.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/spark-nlp-starter-fix.jar with timestamp 1596713653999
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/spark-nlp-starter-fix.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp3059685809950020822.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/spark-nlp-starter-fix.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.hadoop_hadoop-aws-3.2.0.jar with timestamp 1596713653985
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.hadoop_hadoop-aws-3.2.0.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp7724619620694174721.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.apache.hadoop_hadoop-aws-3.2.0.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.avro_avro-1.8.2.jar with timestamp 1596713653960
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.avro_avro-1.8.2.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp2597833214563561673.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.apache.avro_avro-1.8.2.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.json4s_json4s-ext_2.11-3.5.3.jar with timestamp 1596713653989
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.json4s_json4s-ext_2.11-3.5.3.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp3607813927168464210.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.json4s_json4s-ext_2.11-3.5.3.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.commons_commons-compress-1.4.1.jar with timestamp 1596713653969
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.commons_commons-compress-1.4.1.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp8105950885137938160.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.apache.commons_commons-compress-1.4.1.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.amazonaws_jmespath-java-1.11.603.jar with timestamp 1596713653992
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.amazonaws_jmespath-java-1.11.603.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp5982290952228833752.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.amazonaws_jmespath-java-1.11.603.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/commons-httpclient_commons-httpclient-3.1.jar with timestamp 1596713653964
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/commons-httpclient_commons-httpclient-3.1.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp8012038605714845508.tmp
2020-08-06 11:34:40 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/commons-httpclient_commons-httpclient-3.1.jar to class loader
2020-08-06 11:34:40 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.tensorflow_libtensorflow_jni-1.15.0.jar with timestamp 1596713653999
2020-08-06 11:34:40 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.tensorflow_libtensorflow_jni-1.15.0.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp3385213927717898620.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.tensorflow_libtensorflow_jni-1.15.0.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.spark-project.hive_hive-exec-1.2.1.spark2.jar with timestamp 1596713653959
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.spark-project.hive_hive-exec-1.2.1.spark2.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp6766030905944933613.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.spark-project.hive_hive-exec-1.2.1.spark2.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.antlr_antlr-runtime-3.4.jar with timestamp 1596713653969
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.antlr_antlr-runtime-3.4.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp4737213518390433877.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.antlr_antlr-runtime-3.4.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/javolution_javolution-5.5.1.jar with timestamp 1596713653968
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/javolution_javolution-5.5.1.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp7638283030149183107.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/javolution_javolution-5.5.1.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.fasterxml.jackson.core_jackson-databind-2.6.7.2.jar with timestamp 1596713653993
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.fasterxml.jackson.core_jackson-databind-2.6.7.2.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp9070381866309530660.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.fasterxml.jackson.core_jackson-databind-2.6.7.2.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.commons_commons-lang3-3.5.jar with timestamp 1596713653967
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.commons_commons-lang3-3.5.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp3007801371670282758.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.apache.commons_commons-lang3-3.5.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/javax.jdo_jdo-api-3.0.1.jar with timestamp 1596713653977
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/javax.jdo_jdo-api-3.0.1.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp319011132559535326.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/javax.jdo_jdo-api-3.0.1.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.thrift_libfb303-0.9.3.jar with timestamp 1596713653966
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.thrift_libfb303-0.9.3.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp8997581063098432225.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.apache.thrift_libfb303-0.9.3.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/net.hydromatic_eigenbase-properties-1.1.5.jar with timestamp 1596713653979
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/net.hydromatic_eigenbase-properties-1.1.5.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp4987390162821140036.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/net.hydromatic_eigenbase-properties-1.1.5.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.navigamez_greex-1.0.jar with timestamp 1596713653989
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.navigamez_greex-1.0.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp2424925394687952497.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.navigamez_greex-1.0.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/commons-cli_commons-cli-1.2.jar with timestamp 1596713653974
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/commons-cli_commons-cli-1.2.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp5495591071244198077.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/commons-cli_commons-cli-1.2.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/net.sf.trove4j_trove4j-3.0.3.jar with timestamp 1596713653989
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/net.sf.trove4j_trove4j-3.0.3.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp1116126279632305302.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/net.sf.trove4j_trove4j-3.0.3.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.google.protobuf_protobuf-java-3.0.0-beta-3.jar with timestamp 1596713653995
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.google.protobuf_protobuf-java-3.0.0-beta-3.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp5835272153374212933.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.google.protobuf_protobuf-java-3.0.0-beta-3.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.jodd_jodd-core-3.5.2.jar with timestamp 1596713653965
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.jodd_jodd-core-3.5.2.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp4250079226051965020.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.jodd_jodd-core-3.5.2.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.slf4j_slf4j-api-1.7.21.jar with timestamp 1596713653996
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.slf4j_slf4j-api-1.7.21.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp2551421282480652694.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.slf4j_slf4j-api-1.7.21.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.httpcomponents_httpcore-4.4.11.jar with timestamp 1596713653991
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.httpcomponents_httpcore-4.4.11.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp533558892828020989.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.apache.httpcomponents_httpcore-4.4.11.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.derby_derby-10.12.1.1.jar with timestamp 1596713653966
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.derby_derby-10.12.1.1.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp6741135270871447799.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.apache.derby_derby-10.12.1.1.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.google.code.findbugs_jsr305-3.0.1.jar with timestamp 1596713653997
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.google.code.findbugs_jsr305-3.0.1.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp8472838798008753024.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.google.code.findbugs_jsr305-3.0.1.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.datanucleus_datanucleus-rdbms-3.2.9.jar with timestamp 1596713653975
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.datanucleus_datanucleus-rdbms-3.2.9.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp5818726258257865824.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.datanucleus_datanucleus-rdbms-3.2.9.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.tukaani_xz-1.5.jar with timestamp 1596713653973
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.tukaani_xz-1.5.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp5952299645409161520.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.tukaani_xz-1.5.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.typesafe_config-1.3.0.jar with timestamp 1596713653980
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.typesafe_config-1.3.0.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp8978228130424653210.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.typesafe_config-1.3.0.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.twitter_parquet-hadoop-bundle-1.6.0.jar with timestamp 1596713653959
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.twitter_parquet-hadoop-bundle-1.6.0.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp8758962923221773122.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.twitter_parquet-hadoop-bundle-1.6.0.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.spark_spark-hive_2.11-2.4.0.jar with timestamp 1596713653958
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.spark_spark-hive_2.11-2.4.0.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp7871330504386639647.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.apache.spark_spark-hive_2.11-2.4.0.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/software.amazon.ion_ion-java-1.0.2.jar with timestamp 1596713653990
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/software.amazon.ion_ion-java-1.0.2.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp399284284355491849.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/software.amazon.ion_ion-java-1.0.2.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/stax_stax-api-1.0.1.jar with timestamp 1596713653971
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/stax_stax-api-1.0.1.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp2336518601169283427.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/stax_stax-api-1.0.1.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.calcite_calcite-core-1.2.0-incubating.jar with timestamp 1596713653965
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.calcite_calcite-core-1.2.0-incubating.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp4676459210669560119.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.apache.calcite_calcite-core-1.2.0-incubating.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.github.universal-automata_liblevenshtein-3.0.0.jar with timestamp 1596713653988
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.github.universal-automata_liblevenshtein-3.0.0.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp8251940691699387775.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.github.universal-automata_liblevenshtein-3.0.0.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/antlr_antlr-2.7.7.jar with timestamp 1596713653972
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/antlr_antlr-2.7.7.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp4229678671935529141.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/antlr_antlr-2.7.7.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.fasterxml.jackson.core_jackson-core-2.6.7.jar with timestamp 1596713653994
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.fasterxml.jackson.core_jackson-core-2.6.7.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp5857046564349160087.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.fasterxml.jackson.core_jackson-core-2.6.7.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/commons-lang_commons-lang-2.6.jar with timestamp 1596713653968
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/commons-lang_commons-lang-2.6.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp7191573013856367509.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/commons-lang_commons-lang-2.6.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.fasterxml.jackson.core_jackson-annotations-2.6.0.jar with timestamp 1596713653993
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.fasterxml.jackson.core_jackson-annotations-2.6.0.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp6351869866324477009.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.fasterxml.jackson.core_jackson-annotations-2.6.0.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/dk.brics.automaton_automaton-1.11-8.jar with timestamp 1596713653997
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/dk.brics.automaton_automaton-1.11-8.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp7442392536611045939.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/dk.brics.automaton_automaton-1.11-8.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.google.code.findbugs_annotations-3.0.1.jar with timestamp 1596713653994
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.google.code.findbugs_annotations-3.0.1.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp8389056813354748253.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.google.code.findbugs_annotations-3.0.1.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.iq80.snappy_snappy-0.2.jar with timestamp 1596713653970
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.iq80.snappy_snappy-0.2.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp7965519115459465025.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.iq80.snappy_snappy-0.2.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.google.protobuf_protobuf-java-util-3.0.0-beta-3.jar with timestamp 1596713653994
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.google.protobuf_protobuf-java-util-3.0.0-beta-3.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp3331649364498684738.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.google.protobuf_protobuf-java-util-3.0.0-beta-3.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/commons-io_commons-io-2.4.jar with timestamp 1596713653967
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/commons-io_commons-io-2.4.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp2902812962316713531.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/commons-io_commons-io-2.4.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.calcite_calcite-linq4j-1.2.0-incubating.jar with timestamp 1596713653978
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.apache.calcite_calcite-linq4j-1.2.0-incubating.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp2188209812694172888.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.apache.calcite_calcite-linq4j-1.2.0-incubating.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.datanucleus_datanucleus-api-jdo-3.2.6.jar with timestamp 1596713653975
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.datanucleus_datanucleus-api-jdo-3.2.6.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp5680148050502219588.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.datanucleus_datanucleus-api-jdo-3.2.6.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.tensorflow_tensorflow-1.15.0.jar with timestamp 1596713653989
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.tensorflow_tensorflow-1.15.0.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp3014276903038667736.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.tensorflow_tensorflow-1.15.0.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.spark-project.hive_hive-metastore-1.2.1.spark2.jar with timestamp 1596713653960
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.spark-project.hive_hive-metastore-1.2.1.spark2.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp3129758325299046356.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.spark-project.hive_hive-metastore-1.2.1.spark2.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.joda_joda-convert-1.8.1.jar with timestamp 1596713653998
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.joda_joda-convert-1.8.1.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp1697673258694735388.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.joda_joda-convert-1.8.1.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/com.amazonaws_aws-java-sdk-core-1.11.603.jar with timestamp 1596713653988
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/com.amazonaws_aws-java-sdk-core-1.11.603.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp6107071291768234075.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/com.amazonaws_aws-java-sdk-core-1.11.603.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/net.jcip_jcip-annotations-1.0.jar with timestamp 1596713653996
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/net.jcip_jcip-annotations-1.0.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp812153425676370789.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/net.jcip_jcip-annotations-1.0.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/log4j_apache-log4j-extras-1.2.17.jar with timestamp 1596713653968
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/log4j_apache-log4j-extras-1.2.17.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp7514372232763386595.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/log4j_apache-log4j-extras-1.2.17.jar to class loader
2020-08-06 11:34:41 INFO  Executor:54 - Fetching spark://be6d9adbe68c:32961/jars/org.codehaus.jackson_jackson-mapper-asl-1.9.13.jar with timestamp 1596713653965
2020-08-06 11:34:41 INFO  Utils:54 - Fetching spark://be6d9adbe68c:32961/jars/org.codehaus.jackson_jackson-mapper-asl-1.9.13.jar to /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/fetchFileTemp1604672601408973243.tmp
2020-08-06 11:34:41 INFO  Executor:54 - Adding file:/tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af/userFiles-a6503f69-0b2d-474b-ac01-1097e3f3ed03/org.codehaus.jackson_jackson-mapper-asl-1.9.13.jar to class loader
2020-08-06 11:34:41 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/glove_100d_en_2.4.0_2.4_1579690104032/metadata/part-00000:0+377
2020-08-06 11:34:42 INFO  Executor:54 - Finished task 0.0 in stage 0.0 (TID 0). 1176 bytes result sent to driver
2020-08-06 11:34:42 INFO  TaskSetManager:54 - Finished task 0.0 in stage 0.0 (TID 0) in 2630 ms on localhost (executor driver) (1/1)
2020-08-06 11:34:42 INFO  TaskSchedulerImpl:54 - Removed TaskSet 0.0, whose tasks have all completed, from pool 
2020-08-06 11:34:42 INFO  DAGScheduler:54 - ResultStage 0 (first at ReadWrite.scala:615) finished in 2.830 s
2020-08-06 11:34:42 INFO  DAGScheduler:54 - Job 0 finished: first at ReadWrite.scala:615, took 2.883488 s
ner_dl download started this may take some time.
2020-08-06 11:34:43 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
2020-08-06 11:34:43 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
2020-08-06 11:34:43 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
2020-08-06 11:34:43 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
2020-08-06 11:34:44 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
2020-08-06 11:34:44 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
2020-08-06 11:34:44 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
2020-08-06 11:34:44 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
Approximate size to download 13.6 MB
2020-08-06 11:34:44 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
2020-08-06 11:34:44 WARN  ApacheUtils:170 - NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
Download done! Loading the resource.
2020-08-06 11:34:46 INFO  MemoryStore:54 - Block broadcast_2 stored as values in memory (estimated size 236.7 KB, free 2004.1 MB)
2020-08-06 11:34:46 INFO  MemoryStore:54 - Block broadcast_2_piece0 stored as bytes in memory (estimated size 22.9 KB, free 2004.1 MB)
2020-08-06 11:34:46 INFO  BlockManagerInfo:54 - Added broadcast_2_piece0 in memory on be6d9adbe68c:36193 (size: 22.9 KB, free: 2004.6 MB)
2020-08-06 11:34:46 INFO  SparkContext:54 - Created broadcast 2 from textFile at ReadWrite.scala:615
2020-08-06 11:34:46 INFO  FileInputFormat:249 - Total input paths to process : 1
2020-08-06 11:34:46 INFO  SparkContext:54 - Starting job: first at ReadWrite.scala:615
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Got job 1 (first at ReadWrite.scala:615) with 1 output partitions
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Final stage: ResultStage 1 (first at ReadWrite.scala:615)
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Parents of final stage: List()
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Missing parents: List()
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Submitting ResultStage 1 (file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/metadata MapPartitionsRDD[3] at textFile at ReadWrite.scala:615), which has no missing parents
2020-08-06 11:34:46 INFO  MemoryStore:54 - Block broadcast_3 stored as values in memory (estimated size 3.5 KB, free 2004.1 MB)
2020-08-06 11:34:46 INFO  MemoryStore:54 - Block broadcast_3_piece0 stored as bytes in memory (estimated size 2.1 KB, free 2004.1 MB)
2020-08-06 11:34:46 INFO  BlockManagerInfo:54 - Added broadcast_3_piece0 in memory on be6d9adbe68c:36193 (size: 2.1 KB, free: 2004.6 MB)
2020-08-06 11:34:46 INFO  SparkContext:54 - Created broadcast 3 from broadcast at DAGScheduler.scala:1161
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 1 (file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/metadata MapPartitionsRDD[3] at textFile at ReadWrite.scala:615) (first 15 tasks are for partitions Vector(0))
2020-08-06 11:34:46 INFO  TaskSchedulerImpl:54 - Adding task set 1.0 with 1 tasks
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 0.0 in stage 1.0 (TID 1, localhost, executor driver, partition 0, PROCESS_LOCAL, 7941 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 0.0 in stage 1.0 (TID 1)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/metadata/part-00000:0+408
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 0.0 in stage 1.0 (TID 1). 1207 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 0.0 in stage 1.0 (TID 1) in 13 ms on localhost (executor driver) (1/1)
2020-08-06 11:34:46 INFO  TaskSchedulerImpl:54 - Removed TaskSet 1.0, whose tasks have all completed, from pool 
2020-08-06 11:34:46 INFO  DAGScheduler:54 - ResultStage 1 (first at ReadWrite.scala:615) finished in 0.032 s
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Job 1 finished: first at ReadWrite.scala:615, took 0.037452 s
2020-08-06 11:34:46 INFO  MemoryStore:54 - Block broadcast_4 stored as values in memory (estimated size 236.7 KB, free 2003.9 MB)
2020-08-06 11:34:46 INFO  MemoryStore:54 - Block broadcast_4_piece0 stored as bytes in memory (estimated size 22.9 KB, free 2003.8 MB)
2020-08-06 11:34:46 INFO  BlockManagerInfo:54 - Added broadcast_4_piece0 in memory on be6d9adbe68c:36193 (size: 22.9 KB, free: 2004.5 MB)
2020-08-06 11:34:46 INFO  SparkContext:54 - Created broadcast 4 from objectFile at Feature.scala:119
2020-08-06 11:34:46 INFO  FileInputFormat:249 - Total input paths to process : 40
2020-08-06 11:34:46 INFO  SparkContext:54 - Starting job: first at Feature.scala:119
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Got job 2 (first at Feature.scala:119) with 1 output partitions
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Final stage: ResultStage 2 (first at Feature.scala:119)
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Parents of final stage: List()
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Missing parents: List()
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Submitting ResultStage 2 (MapPartitionsRDD[5] at objectFile at Feature.scala:119), which has no missing parents
2020-08-06 11:34:46 INFO  MemoryStore:54 - Block broadcast_5 stored as values in memory (estimated size 3.7 KB, free 2003.8 MB)
2020-08-06 11:34:46 INFO  MemoryStore:54 - Block broadcast_5_piece0 stored as bytes in memory (estimated size 2.2 KB, free 2003.8 MB)
2020-08-06 11:34:46 INFO  BlockManagerInfo:54 - Added broadcast_5_piece0 in memory on be6d9adbe68c:36193 (size: 2.2 KB, free: 2004.5 MB)
2020-08-06 11:34:46 INFO  SparkContext:54 - Created broadcast 5 from broadcast at DAGScheduler.scala:1161
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 2 (MapPartitionsRDD[5] at objectFile at Feature.scala:119) (first 15 tasks are for partitions Vector(0))
2020-08-06 11:34:46 INFO  TaskSchedulerImpl:54 - Adding task set 2.0 with 1 tasks
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 0.0 in stage 2.0 (TID 2, localhost, executor driver, partition 0, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 0.0 in stage 2.0 (TID 2)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00011:0+95
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 0.0 in stage 2.0 (TID 2). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 0.0 in stage 2.0 (TID 2) in 35 ms on localhost (executor driver) (1/1)
2020-08-06 11:34:46 INFO  TaskSchedulerImpl:54 - Removed TaskSet 2.0, whose tasks have all completed, from pool 
2020-08-06 11:34:46 INFO  DAGScheduler:54 - ResultStage 2 (first at Feature.scala:119) finished in 0.047 s
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Job 2 finished: first at Feature.scala:119, took 0.050564 s
2020-08-06 11:34:46 INFO  SparkContext:54 - Starting job: first at Feature.scala:119
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Got job 3 (first at Feature.scala:119) with 4 output partitions
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Final stage: ResultStage 3 (first at Feature.scala:119)
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Parents of final stage: List()
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Missing parents: List()
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Submitting ResultStage 3 (MapPartitionsRDD[5] at objectFile at Feature.scala:119), which has no missing parents
2020-08-06 11:34:46 INFO  MemoryStore:54 - Block broadcast_6 stored as values in memory (estimated size 3.7 KB, free 2003.8 MB)
2020-08-06 11:34:46 INFO  MemoryStore:54 - Block broadcast_6_piece0 stored as bytes in memory (estimated size 2.2 KB, free 2003.8 MB)
2020-08-06 11:34:46 INFO  BlockManagerInfo:54 - Added broadcast_6_piece0 in memory on be6d9adbe68c:36193 (size: 2.2 KB, free: 2004.5 MB)
2020-08-06 11:34:46 INFO  SparkContext:54 - Created broadcast 6 from broadcast at DAGScheduler.scala:1161
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Submitting 4 missing tasks from ResultStage 3 (MapPartitionsRDD[5] at objectFile at Feature.scala:119) (first 15 tasks are for partitions Vector(1, 2, 3, 4))
2020-08-06 11:34:46 INFO  TaskSchedulerImpl:54 - Adding task set 3.0 with 4 tasks
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 0.0 in stage 3.0 (TID 3, localhost, executor driver, partition 1, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 1.0 in stage 3.0 (TID 4, localhost, executor driver, partition 2, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 2.0 in stage 3.0 (TID 5, localhost, executor driver, partition 3, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 3.0 in stage 3.0 (TID 6, localhost, executor driver, partition 4, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 0.0 in stage 3.0 (TID 3)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 1.0 in stage 3.0 (TID 4)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 3.0 in stage 3.0 (TID 6)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 2.0 in stage 3.0 (TID 5)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00001:0+95
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00036:0+95
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 0.0 in stage 3.0 (TID 3). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00009:0+95
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00024:0+95
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 0.0 in stage 3.0 (TID 3) in 33 ms on localhost (executor driver) (1/4)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 1.0 in stage 3.0 (TID 4). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 1.0 in stage 3.0 (TID 4) in 35 ms on localhost (executor driver) (2/4)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 2.0 in stage 3.0 (TID 5). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 3.0 in stage 3.0 (TID 6). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 3.0 in stage 3.0 (TID 6) in 39 ms on localhost (executor driver) (3/4)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 2.0 in stage 3.0 (TID 5) in 39 ms on localhost (executor driver) (4/4)
2020-08-06 11:34:46 INFO  TaskSchedulerImpl:54 - Removed TaskSet 3.0, whose tasks have all completed, from pool 
2020-08-06 11:34:46 INFO  DAGScheduler:54 - ResultStage 3 (first at Feature.scala:119) finished in 0.051 s
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Job 3 finished: first at Feature.scala:119, took 0.056233 s
2020-08-06 11:34:46 INFO  SparkContext:54 - Starting job: first at Feature.scala:119
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Got job 4 (first at Feature.scala:119) with 20 output partitions
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Final stage: ResultStage 4 (first at Feature.scala:119)
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Parents of final stage: List()
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Missing parents: List()
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Submitting ResultStage 4 (MapPartitionsRDD[5] at objectFile at Feature.scala:119), which has no missing parents
2020-08-06 11:34:46 INFO  MemoryStore:54 - Block broadcast_7 stored as values in memory (estimated size 3.7 KB, free 2003.8 MB)
2020-08-06 11:34:46 INFO  MemoryStore:54 - Block broadcast_7_piece0 stored as bytes in memory (estimated size 2.2 KB, free 2003.8 MB)
2020-08-06 11:34:46 INFO  BlockManagerInfo:54 - Added broadcast_7_piece0 in memory on be6d9adbe68c:36193 (size: 2.2 KB, free: 2004.5 MB)
2020-08-06 11:34:46 INFO  SparkContext:54 - Created broadcast 7 from broadcast at DAGScheduler.scala:1161
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Submitting 20 missing tasks from ResultStage 4 (MapPartitionsRDD[5] at objectFile at Feature.scala:119) (first 15 tasks are for partitions Vector(5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19))
2020-08-06 11:34:46 INFO  TaskSchedulerImpl:54 - Adding task set 4.0 with 20 tasks
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 0.0 in stage 4.0 (TID 7, localhost, executor driver, partition 5, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 1.0 in stage 4.0 (TID 8, localhost, executor driver, partition 6, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 2.0 in stage 4.0 (TID 9, localhost, executor driver, partition 7, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 3.0 in stage 4.0 (TID 10, localhost, executor driver, partition 8, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 2.0 in stage 4.0 (TID 9)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 0.0 in stage 4.0 (TID 7)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 3.0 in stage 4.0 (TID 10)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 1.0 in stage 4.0 (TID 8)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00003:0+95
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00034:0+95
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00033:0+95
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00014:0+95
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 2.0 in stage 4.0 (TID 9). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 4.0 in stage 4.0 (TID 11, localhost, executor driver, partition 9, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 1.0 in stage 4.0 (TID 8). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 0.0 in stage 4.0 (TID 7). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 2.0 in stage 4.0 (TID 9) in 16 ms on localhost (executor driver) (1/20)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 3.0 in stage 4.0 (TID 10). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  Executor:54 - Running task 4.0 in stage 4.0 (TID 11)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 5.0 in stage 4.0 (TID 12, localhost, executor driver, partition 10, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 5.0 in stage 4.0 (TID 12)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 6.0 in stage 4.0 (TID 13, localhost, executor driver, partition 11, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 7.0 in stage 4.0 (TID 14, localhost, executor driver, partition 12, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 6.0 in stage 4.0 (TID 13)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 7.0 in stage 4.0 (TID 14)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00015:0+95
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00013:0+95
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00004:0+95
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 1.0 in stage 4.0 (TID 8) in 31 ms on localhost (executor driver) (2/20)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 3.0 in stage 4.0 (TID 10) in 31 ms on localhost (executor driver) (3/20)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 0.0 in stage 4.0 (TID 7) in 34 ms on localhost (executor driver) (4/20)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00017:0+95
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 5.0 in stage 4.0 (TID 12). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 8.0 in stage 4.0 (TID 15, localhost, executor driver, partition 13, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 5.0 in stage 4.0 (TID 12) in 17 ms on localhost (executor driver) (5/20)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 4.0 in stage 4.0 (TID 11). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  Executor:54 - Running task 8.0 in stage 4.0 (TID 15)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 9.0 in stage 4.0 (TID 16, localhost, executor driver, partition 14, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 7.0 in stage 4.0 (TID 14). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 4.0 in stage 4.0 (TID 11) in 28 ms on localhost (executor driver) (6/20)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 6.0 in stage 4.0 (TID 13). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 10.0 in stage 4.0 (TID 17, localhost, executor driver, partition 15, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 10.0 in stage 4.0 (TID 17)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00006:0+95
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 8.0 in stage 4.0 (TID 15). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  Executor:54 - Running task 9.0 in stage 4.0 (TID 16)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 7.0 in stage 4.0 (TID 14) in 19 ms on localhost (executor driver) (7/20)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 11.0 in stage 4.0 (TID 18, localhost, executor driver, partition 16, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 11.0 in stage 4.0 (TID 18)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00008:0+95
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 12.0 in stage 4.0 (TID 19, localhost, executor driver, partition 17, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 8.0 in stage 4.0 (TID 15) in 23 ms on localhost (executor driver) (8/20)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 12.0 in stage 4.0 (TID 19)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00002:0+95
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00032:0+95
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 6.0 in stage 4.0 (TID 13) in 39 ms on localhost (executor driver) (9/20)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 10.0 in stage 4.0 (TID 17). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 13.0 in stage 4.0 (TID 20, localhost, executor driver, partition 18, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 13.0 in stage 4.0 (TID 20)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 10.0 in stage 4.0 (TID 17) in 24 ms on localhost (executor driver) (10/20)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 11.0 in stage 4.0 (TID 18). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00023:0+95
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 14.0 in stage 4.0 (TID 21, localhost, executor driver, partition 19, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 14.0 in stage 4.0 (TID 21)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 11.0 in stage 4.0 (TID 18) in 13 ms on localhost (executor driver) (11/20)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 13.0 in stage 4.0 (TID 20). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 15.0 in stage 4.0 (TID 22, localhost, executor driver, partition 20, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 9.0 in stage 4.0 (TID 16). 829 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 13.0 in stage 4.0 (TID 20) in 9 ms on localhost (executor driver) (12/20)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 15.0 in stage 4.0 (TID 22)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00019:0+95
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 12.0 in stage 4.0 (TID 19). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 16.0 in stage 4.0 (TID 23, localhost, executor driver, partition 21, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00026:0+95
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 15.0 in stage 4.0 (TID 22). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00038:0+95
2020-08-06 11:34:46 INFO  Executor:54 - Running task 16.0 in stage 4.0 (TID 23)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 17.0 in stage 4.0 (TID 24, localhost, executor driver, partition 22, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 17.0 in stage 4.0 (TID 24)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00005:0+95
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 18.0 in stage 4.0 (TID 25, localhost, executor driver, partition 23, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 12.0 in stage 4.0 (TID 19) in 45 ms on localhost (executor driver) (13/20)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 18.0 in stage 4.0 (TID 25)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 14.0 in stage 4.0 (TID 21). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 16.0 in stage 4.0 (TID 23). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 9.0 in stage 4.0 (TID 16) in 66 ms on localhost (executor driver) (14/20)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 19.0 in stage 4.0 (TID 26, localhost, executor driver, partition 24, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 15.0 in stage 4.0 (TID 22) in 35 ms on localhost (executor driver) (15/20)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 14.0 in stage 4.0 (TID 21) in 39 ms on localhost (executor driver) (16/20)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 16.0 in stage 4.0 (TID 23) in 27 ms on localhost (executor driver) (17/20)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00018:0+95
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00031:0+95
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 18.0 in stage 4.0 (TID 25). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  Executor:54 - Running task 19.0 in stage 4.0 (TID 26)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00010:0+95
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 17.0 in stage 4.0 (TID 24). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 18.0 in stage 4.0 (TID 25) in 20 ms on localhost (executor driver) (18/20)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 17.0 in stage 4.0 (TID 24) in 24 ms on localhost (executor driver) (19/20)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 19.0 in stage 4.0 (TID 26). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 19.0 in stage 4.0 (TID 26) in 19 ms on localhost (executor driver) (20/20)
2020-08-06 11:34:46 INFO  TaskSchedulerImpl:54 - Removed TaskSet 4.0, whose tasks have all completed, from pool 
2020-08-06 11:34:46 INFO  DAGScheduler:54 - ResultStage 4 (first at Feature.scala:119) finished in 0.138 s
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Job 4 finished: first at Feature.scala:119, took 0.142857 s
2020-08-06 11:34:46 INFO  SparkContext:54 - Starting job: first at Feature.scala:119
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Got job 5 (first at Feature.scala:119) with 15 output partitions
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Final stage: ResultStage 5 (first at Feature.scala:119)
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Parents of final stage: List()
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Missing parents: List()
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Submitting ResultStage 5 (MapPartitionsRDD[5] at objectFile at Feature.scala:119), which has no missing parents
2020-08-06 11:34:46 INFO  MemoryStore:54 - Block broadcast_8 stored as values in memory (estimated size 3.7 KB, free 2003.8 MB)
2020-08-06 11:34:46 INFO  MemoryStore:54 - Block broadcast_8_piece0 stored as bytes in memory (estimated size 2.2 KB, free 2003.8 MB)
2020-08-06 11:34:46 INFO  BlockManagerInfo:54 - Added broadcast_8_piece0 in memory on be6d9adbe68c:36193 (size: 2.2 KB, free: 2004.5 MB)
2020-08-06 11:34:46 INFO  SparkContext:54 - Created broadcast 8 from broadcast at DAGScheduler.scala:1161
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Submitting 15 missing tasks from ResultStage 5 (MapPartitionsRDD[5] at objectFile at Feature.scala:119) (first 15 tasks are for partitions Vector(25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39))
2020-08-06 11:34:46 INFO  TaskSchedulerImpl:54 - Adding task set 5.0 with 15 tasks
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 0.0 in stage 5.0 (TID 27, localhost, executor driver, partition 25, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 1.0 in stage 5.0 (TID 28, localhost, executor driver, partition 26, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 2.0 in stage 5.0 (TID 29, localhost, executor driver, partition 27, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 3.0 in stage 5.0 (TID 30, localhost, executor driver, partition 28, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 0.0 in stage 5.0 (TID 27)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 2.0 in stage 5.0 (TID 29)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 1.0 in stage 5.0 (TID 28)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 3.0 in stage 5.0 (TID 30)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00029:0+95
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00037:0+95
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00016:0+95
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 1.0 in stage 5.0 (TID 28). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 4.0 in stage 5.0 (TID 31, localhost, executor driver, partition 29, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00030:0+95
2020-08-06 11:34:46 INFO  Executor:54 - Running task 4.0 in stage 5.0 (TID 31)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 3.0 in stage 5.0 (TID 30). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 1.0 in stage 5.0 (TID 28) in 13 ms on localhost (executor driver) (1/15)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 0.0 in stage 5.0 (TID 27). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 2.0 in stage 5.0 (TID 29). 829 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 5.0 in stage 5.0 (TID 32, localhost, executor driver, partition 30, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00000:0+95
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 6.0 in stage 5.0 (TID 33, localhost, executor driver, partition 31, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 0.0 in stage 5.0 (TID 27) in 25 ms on localhost (executor driver) (2/15)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 3.0 in stage 5.0 (TID 30) in 24 ms on localhost (executor driver) (3/15)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 7.0 in stage 5.0 (TID 34, localhost, executor driver, partition 32, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 7.0 in stage 5.0 (TID 34)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00012:0+95
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 4.0 in stage 5.0 (TID 31). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  Executor:54 - Running task 5.0 in stage 5.0 (TID 32)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 6.0 in stage 5.0 (TID 33)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 7.0 in stage 5.0 (TID 34). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 2.0 in stage 5.0 (TID 29) in 30 ms on localhost (executor driver) (4/15)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 8.0 in stage 5.0 (TID 35, localhost, executor driver, partition 33, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 4.0 in stage 5.0 (TID 31) in 26 ms on localhost (executor driver) (5/15)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00007:0+95
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 9.0 in stage 5.0 (TID 36, localhost, executor driver, partition 34, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 8.0 in stage 5.0 (TID 35)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 7.0 in stage 5.0 (TID 34) in 10 ms on localhost (executor driver) (6/15)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 9.0 in stage 5.0 (TID 36)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00020:0+95
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 5.0 in stage 5.0 (TID 32). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00027:0+95
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00025:0+95
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 10.0 in stage 5.0 (TID 37, localhost, executor driver, partition 35, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 5.0 in stage 5.0 (TID 32) in 28 ms on localhost (executor driver) (7/15)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 10.0 in stage 5.0 (TID 37)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 6.0 in stage 5.0 (TID 33). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 11.0 in stage 5.0 (TID 38, localhost, executor driver, partition 36, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 6.0 in stage 5.0 (TID 33) in 29 ms on localhost (executor driver) (8/15)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 11.0 in stage 5.0 (TID 38)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00022:0+95
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00028:0+95
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 9.0 in stage 5.0 (TID 36). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 10.0 in stage 5.0 (TID 37). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 8.0 in stage 5.0 (TID 35). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 12.0 in stage 5.0 (TID 39, localhost, executor driver, partition 37, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 9.0 in stage 5.0 (TID 36) in 25 ms on localhost (executor driver) (9/15)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 13.0 in stage 5.0 (TID 40, localhost, executor driver, partition 38, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 13.0 in stage 5.0 (TID 40)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Starting task 14.0 in stage 5.0 (TID 41, localhost, executor driver, partition 39, PROCESS_LOCAL, 7953 bytes)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 10.0 in stage 5.0 (TID 37) in 21 ms on localhost (executor driver) (10/15)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 8.0 in stage 5.0 (TID 35) in 31 ms on localhost (executor driver) (11/15)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 11.0 in stage 5.0 (TID 38). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  Executor:54 - Running task 14.0 in stage 5.0 (TID 41)
2020-08-06 11:34:46 INFO  Executor:54 - Running task 12.0 in stage 5.0 (TID 39)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00039:0+2831
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 11.0 in stage 5.0 (TID 38) in 18 ms on localhost (executor driver) (12/15)
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00035:0+95
2020-08-06 11:34:46 INFO  HadoopRDD:54 - Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00021:0+95
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 14.0 in stage 5.0 (TID 41). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 12.0 in stage 5.0 (TID 39). 786 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 12.0 in stage 5.0 (TID 39) in 17 ms on localhost (executor driver) (13/15)
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 14.0 in stage 5.0 (TID 41) in 16 ms on localhost (executor driver) (14/15)
2020-08-06 11:34:46 INFO  Executor:54 - Finished task 13.0 in stage 5.0 (TID 40). 3487 bytes result sent to driver
2020-08-06 11:34:46 INFO  TaskSetManager:54 - Finished task 13.0 in stage 5.0 (TID 40) in 48 ms on localhost (executor driver) (15/15)
2020-08-06 11:34:46 INFO  TaskSchedulerImpl:54 - Removed TaskSet 5.0, whose tasks have all completed, from pool 
2020-08-06 11:34:46 INFO  DAGScheduler:54 - ResultStage 5 (first at Feature.scala:119) finished in 0.135 s
2020-08-06 11:34:46 INFO  DAGScheduler:54 - Job 5 finished: first at Feature.scala:119, took 0.140753 s
2020-08-06 11:34:46 INFO  MemoryStore:54 - Block broadcast_9 stored as values in memory (estimated size 8.4 KB, free 2003.8 MB)
2020-08-06 11:34:46 INFO  MemoryStore:54 - Block broadcast_9_piece0 stored as bytes in memory (estimated size 900.0 B, free 2003.8 MB)
2020-08-06 11:34:46 INFO  BlockManagerInfo:54 - Added broadcast_9_piece0 in memory on be6d9adbe68c:36193 (size: 900.0 B, free: 2004.5 MB)
2020-08-06 11:34:46 INFO  SparkContext:54 - Created broadcast 9 from broadcast at Feature.scala:86
Exception in thread "main" java.lang.UnsatisfiedLinkError: /tmp/tensorflow_native_libraries-1596713687183-0/libtensorflow_jni.so: Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /tmp/tensorflow_native_libraries-1596713687183-0/libtensorflow_jni.so)
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
    at java.lang.Runtime.load0(Runtime.java:809)
    at java.lang.System.load(System.java:1086)
    at org.tensorflow.NativeLibrary.load(NativeLibrary.java:100)
    at org.tensorflow.TensorFlow.init(TensorFlow.java:67)
    at org.tensorflow.TensorFlow.<clinit>(TensorFlow.java:82)
    at org.tensorflow.Graph.<clinit>(Graph.java:479)
    at com.johnsnowlabs.ml.tensorflow.TensorflowWrapper$.readGraph(TensorflowWrapper.scala:207)
    at com.johnsnowlabs.ml.tensorflow.TensorflowWrapper$.read(TensorflowWrapper.scala:252)
    at com.johnsnowlabs.ml.tensorflow.ReadTensorflowModel$class.readTensorflowModel(TensorflowSerializeModel.scala:97)
    at com.johnsnowlabs.nlp.annotators.ner.dl.NerDLModel$.readTensorflowModel(NerDLModel.scala:227)
    at com.johnsnowlabs.nlp.annotators.ner.dl.ReadsNERGraph$class.readNerGraph(NerDLModel.scala:207)
    at com.johnsnowlabs.nlp.annotators.ner.dl.NerDLModel$.readNerGraph(NerDLModel.scala:227)
    at com.johnsnowlabs.nlp.annotators.ner.dl.ReadsNERGraph$$anonfun$2.apply(NerDLModel.scala:211)
    at com.johnsnowlabs.nlp.annotators.ner.dl.ReadsNERGraph$$anonfun$2.apply(NerDLModel.scala:211)
    at com.johnsnowlabs.nlp.ParamsAndFeaturesReadable$$anonfun$com$johnsnowlabs$nlp$ParamsAndFeaturesReadable$$onRead$1.apply(ParamsAndFeaturesReadable.scala:31)
    at com.johnsnowlabs.nlp.ParamsAndFeaturesReadable$$anonfun$com$johnsnowlabs$nlp$ParamsAndFeaturesReadable$$onRead$1.apply(ParamsAndFeaturesReadable.scala:30)
    at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
    at com.johnsnowlabs.nlp.ParamsAndFeaturesReadable$class.com$johnsnowlabs$nlp$ParamsAndFeaturesReadable$$onRead(ParamsAndFeaturesReadable.scala:30)
    at com.johnsnowlabs.nlp.ParamsAndFeaturesReadable$$anonfun$read$1.apply(ParamsAndFeaturesReadable.scala:41)
    at com.johnsnowlabs.nlp.ParamsAndFeaturesReadable$$anonfun$read$1.apply(ParamsAndFeaturesReadable.scala:41)
    at com.johnsnowlabs.nlp.FeaturesReader.load(ParamsAndFeaturesReadable.scala:19)
    at com.johnsnowlabs.nlp.FeaturesReader.load(ParamsAndFeaturesReadable.scala:8)
    at com.johnsnowlabs.nlp.pretrained.ResourceDownloader$.downloadModel(ResourceDownloader.scala:360)
    at com.johnsnowlabs.nlp.pretrained.ResourceDownloader$.downloadModel(ResourceDownloader.scala:354)
    at com.johnsnowlabs.nlp.annotators.ner.dl.ReadablePretrainedNerDL$class.pretrained(NerDLModel.scala:218)
    at com.johnsnowlabs.nlp.annotator.package$NerDLModel$.pretrained(annotator.scala:121)
    at com.johnsnowlabs.nlp.annotators.ner.dl.ReadablePretrainedNerDL$class.pretrained(NerDLModel.scala:223)
    at com.johnsnowlabs.nlp.annotator.package$NerDLModel$.pretrained(annotator.scala:121)
    at Main$.main(Main.scala:31)
    at Main.main(Main.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
    at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
    at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
    at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
    at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
    at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
2020-08-06 11:34:48 INFO  SparkContext:54 - Invoking stop() from shutdown hook
2020-08-06 11:34:48 INFO  AbstractConnector:318 - Stopped Spark@10c72a6f{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
2020-08-06 11:34:48 INFO  SparkUI:54 - Stopped Spark web UI at http://be6d9adbe68c:4040
2020-08-06 11:34:48 INFO  MapOutputTrackerMasterEndpoint:54 - MapOutputTrackerMasterEndpoint stopped!
2020-08-06 11:34:48 INFO  MemoryStore:54 - MemoryStore cleared
2020-08-06 11:34:48 INFO  BlockManager:54 - BlockManager stopped
2020-08-06 11:34:48 INFO  BlockManagerMaster:54 - BlockManagerMaster stopped
2020-08-06 11:34:48 INFO  OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:54 - OutputCommitCoordinator stopped!
2020-08-06 11:34:48 INFO  SparkContext:54 - Successfully stopped SparkContext
2020-08-06 11:34:48 INFO  ShutdownHookManager:54 - Shutdown hook called
2020-08-06 11:34:48 INFO  ShutdownHookManager:54 - Deleting directory /tmp/spark-968b8f4e-5761-4223-8e3c-a7daf7bdaf14
2020-08-06 11:34:48 INFO  ShutdownHookManager:54 - Deleting directory /tmp/spark-ae90f755-e5d3-4632-bf9e-7e060b6837af

I will try to create new images with Ubuntu, it might take some time so I will leave this comment, maybe it gives you some other ideas.

maziyarpanahi commented 4 years ago

Thanks @miloradtrninic this is very helpful. I'll share this and wait for you for the Ubuntu test.

miloradtrninic commented 4 years ago

When I use ubuntu:18.04 as a base image I get same error which started this discussion Here is the new log file from core dump https://github.com/miloradtrninic/entity/blob/master/hs_err_pid66.log.txt

Edit: Can you give me some environment example where Spark NLP works? OS, pySpark over spark etc... 

docker exec spark-master /spark/bin/spark-submit --packages org.apache.spark:spark-hive_2.11:2.4.0,com.johnsnowlabs.nlp:spark-nlp_2.11:2.5.4 --driver-memory 4g --class "Main" /deploy/spark-nlp-starter-fix.jar
Ivy Default Cache set to: /root/.ivy2/cache
The jars for the packages stored in: /root/.ivy2/jars
:: loading settings :: url = jar:file:/spark/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
org.apache.spark#spark-hive_2.11 added as a dependency
com.johnsnowlabs.nlp#spark-nlp_2.11 added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent-8c906343-c23f-41b7-ab1d-64fd3eccb6f2;1.0
    confs: [default]
    found org.apache.spark#spark-hive_2.11;2.4.0 in central
    found com.twitter#parquet-hadoop-bundle;1.6.0 in central
    found org.spark-project.hive#hive-exec;1.2.1.spark2 in central
    found commons-io#commons-io;2.4 in central
    found org.apache.commons#commons-lang3;3.5 in central
    found commons-lang#commons-lang;2.6 in central
    found javolution#javolution;5.5.1 in central
    found log4j#apache-log4j-extras;1.2.17 in central
    found org.antlr#antlr-runtime;3.4 in central
    found org.antlr#stringtemplate;3.2.1 in central
    found antlr#antlr;2.7.7 in central
    found org.antlr#ST4;4.0.4 in central
    found org.apache.avro#avro;1.8.2 in central
    found org.codehaus.jackson#jackson-core-asl;1.9.13 in central
    found org.codehaus.jackson#jackson-mapper-asl;1.9.13 in central
    found com.thoughtworks.paranamer#paranamer;2.8 in central
    found org.xerial.snappy#snappy-java;1.1.7.1 in central
    found org.apache.commons#commons-compress;1.4.1 in central
    found org.tukaani#xz;1.5 in central
    found org.slf4j#slf4j-api;1.7.16 in central
    found org.apache.ivy#ivy;2.4.0 in central
    found org.jodd#jodd-core;3.5.2 in central
    found org.datanucleus#datanucleus-core;3.2.10 in central
    found org.apache.calcite#calcite-avatica;1.2.0-incubating in central
    found com.googlecode.javaewah#JavaEWAH;0.3.2 in central
    found org.iq80.snappy#snappy;0.2 in central
    found stax#stax-api;1.0.1 in central
    found net.sf.opencsv#opencsv;2.3 in central
    found org.spark-project.hive#hive-metastore;1.2.1.spark2 in central
    found com.jolbox#bonecp;0.8.0.RELEASE in central
    found commons-cli#commons-cli;1.2 in central
    found commons-logging#commons-logging;1.1.3 in central
    found org.apache.derby#derby;10.12.1.1 in central
    found org.datanucleus#datanucleus-api-jdo;3.2.6 in central
    found org.datanucleus#datanucleus-rdbms;3.2.9 in central
    found commons-pool#commons-pool;1.5.4 in central
    found commons-dbcp#commons-dbcp;1.4 in central
    found javax.jdo#jdo-api;3.0.1 in central
    found javax.transaction#jta;1.1 in central
    found org.apache.avro#avro-mapred;1.8.2 in central
    found org.apache.avro#avro-ipc;1.8.2 in central
    found commons-codec#commons-codec;1.10 in central
    found commons-httpclient#commons-httpclient;3.1 in central
    found org.apache.calcite#calcite-core;1.2.0-incubating in central
    found org.apache.calcite#calcite-linq4j;1.2.0-incubating in central
    found net.hydromatic#eigenbase-properties;1.1.5 in central
    found org.apache.httpcomponents#httpclient;4.5.6 in central
    found org.apache.httpcomponents#httpcore;4.4.10 in central
    found commons-logging#commons-logging;1.2 in central
    found joda-time#joda-time;2.9.3 in central
    found com.google.code.findbugs#jsr305;1.3.9 in central
    found org.apache.thrift#libthrift;0.9.3 in central
    found org.apache.thrift#libfb303;0.9.3 in central
    found org.spark-project.spark#unused;1.0.0 in central
    found com.johnsnowlabs.nlp#spark-nlp_2.11;2.5.4 in central
    found com.typesafe#config;1.3.0 in central
    found org.rocksdb#rocksdbjni;6.5.3 in central
    found org.apache.hadoop#hadoop-aws;3.2.0 in central
    found com.amazonaws#aws-java-sdk-core;1.11.603 in central
    found org.apache.httpcomponents#httpclient;4.5.9 in central
    found org.apache.httpcomponents#httpcore;4.4.11 in central
    found commons-codec#commons-codec;1.11 in central
    found software.amazon.ion#ion-java;1.0.2 in central
    found com.fasterxml.jackson.dataformat#jackson-dataformat-cbor;2.6.7 in central
    found com.amazonaws#aws-java-sdk-s3;1.11.603 in central
    found com.amazonaws#aws-java-sdk-kms;1.11.603 in central
    found com.amazonaws#jmespath-java;1.11.603 in central
    found com.fasterxml.jackson.core#jackson-databind;2.6.7.2 in central
    found com.fasterxml.jackson.core#jackson-annotations;2.6.0 in central
    found com.fasterxml.jackson.core#jackson-core;2.6.7 in central
    found com.github.universal-automata#liblevenshtein;3.0.0 in central
    found com.google.code.findbugs#annotations;3.0.1 in central
    found net.jcip#jcip-annotations;1.0 in central
    found com.google.code.findbugs#jsr305;3.0.1 in central
    found com.google.protobuf#protobuf-java-util;3.0.0-beta-3 in central
    found com.google.protobuf#protobuf-java;3.0.0-beta-3 in central
    found com.google.code.gson#gson;2.3 in central
    found it.unimi.dsi#fastutil;7.0.12 in central
    found org.projectlombok#lombok;1.16.8 in central
    found org.slf4j#slf4j-api;1.7.21 in central
    found com.navigamez#greex;1.0 in central
    found dk.brics.automaton#automaton;1.11-8 in central
    found org.json4s#json4s-ext_2.11;3.5.3 in central
    found joda-time#joda-time;2.9.5 in central
    found org.joda#joda-convert;1.8.1 in central
    found org.tensorflow#tensorflow;1.15.0 in central
    found org.tensorflow#libtensorflow;1.15.0 in central
    found org.tensorflow#libtensorflow_jni;1.15.0 in central
    found net.sf.trove4j#trove4j;3.0.3 in central
downloading https://repo1.maven.org/maven2/org/apache/spark/spark-hive_2.11/2.4.0/spark-hive_2.11-2.4.0.jar ...
    [SUCCESSFUL ] org.apache.spark#spark-hive_2.11;2.4.0!spark-hive_2.11.jar (90ms)
downloading https://repo1.maven.org/maven2/com/johnsnowlabs/nlp/spark-nlp_2.11/2.5.4/spark-nlp_2.11-2.5.4.jar ...
    [SUCCESSFUL ] com.johnsnowlabs.nlp#spark-nlp_2.11;2.5.4!spark-nlp_2.11.jar (663ms)
downloading https://repo1.maven.org/maven2/com/twitter/parquet-hadoop-bundle/1.6.0/parquet-hadoop-bundle-1.6.0.jar ...
    [SUCCESSFUL ] com.twitter#parquet-hadoop-bundle;1.6.0!parquet-hadoop-bundle.jar (77ms)
downloading https://repo1.maven.org/maven2/org/spark-project/hive/hive-exec/1.2.1.spark2/hive-exec-1.2.1.spark2.jar ...
    [SUCCESSFUL ] org.spark-project.hive#hive-exec;1.2.1.spark2!hive-exec.jar (252ms)
downloading https://repo1.maven.org/maven2/org/spark-project/hive/hive-metastore/1.2.1.spark2/hive-metastore-1.2.1.spark2.jar ...
    [SUCCESSFUL ] org.spark-project.hive#hive-metastore;1.2.1.spark2!hive-metastore.jar (180ms)
downloading https://repo1.maven.org/maven2/org/apache/avro/avro/1.8.2/avro-1.8.2.jar ...
    [SUCCESSFUL ] org.apache.avro#avro;1.8.2!avro.jar(bundle) (83ms)
downloading https://repo1.maven.org/maven2/org/apache/avro/avro-mapred/1.8.2/avro-mapred-1.8.2-hadoop2.jar ...
    [SUCCESSFUL ] org.apache.avro#avro-mapred;1.8.2!avro-mapred.jar (37ms)
downloading https://repo1.maven.org/maven2/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar ...
    [SUCCESSFUL ] commons-httpclient#commons-httpclient;3.1!commons-httpclient.jar (29ms)
downloading https://repo1.maven.org/maven2/org/apache/calcite/calcite-avatica/1.2.0-incubating/calcite-avatica-1.2.0-incubating.jar ...
    [SUCCESSFUL ] org.apache.calcite#calcite-avatica;1.2.0-incubating!calcite-avatica.jar (25ms)
downloading https://repo1.maven.org/maven2/org/apache/calcite/calcite-core/1.2.0-incubating/calcite-core-1.2.0-incubating.jar ...
    [SUCCESSFUL ] org.apache.calcite#calcite-core;1.2.0-incubating!calcite-core.jar (169ms)
downloading https://repo1.maven.org/maven2/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar ...
    [SUCCESSFUL ] org.codehaus.jackson#jackson-mapper-asl;1.9.13!jackson-mapper-asl.jar (47ms)
downloading https://repo1.maven.org/maven2/org/jodd/jodd-core/3.5.2/jodd-core-3.5.2.jar ...
    [SUCCESSFUL ] org.jodd#jodd-core;3.5.2!jodd-core.jar (35ms)
downloading https://repo1.maven.org/maven2/org/datanucleus/datanucleus-core/3.2.10/datanucleus-core-3.2.10.jar ...
    [SUCCESSFUL ] org.datanucleus#datanucleus-core;3.2.10!datanucleus-core.jar (94ms)
downloading https://repo1.maven.org/maven2/org/apache/thrift/libthrift/0.9.3/libthrift-0.9.3.jar ...
    [SUCCESSFUL ] org.apache.thrift#libthrift;0.9.3!libthrift.jar (23ms)
downloading https://repo1.maven.org/maven2/org/apache/thrift/libfb303/0.9.3/libfb303-0.9.3.jar ...
    [SUCCESSFUL ] org.apache.thrift#libfb303;0.9.3!libfb303.jar (38ms)
downloading https://repo1.maven.org/maven2/org/apache/derby/derby/10.12.1.1/derby-10.12.1.1.jar ...
    [SUCCESSFUL ] org.apache.derby#derby;10.12.1.1!derby.jar (158ms)
downloading https://repo1.maven.org/maven2/org/spark-project/spark/unused/1.0.0/unused-1.0.0.jar ...
    [SUCCESSFUL ] org.spark-project.spark#unused;1.0.0!unused.jar (15ms)
downloading https://repo1.maven.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.jar ...
    [SUCCESSFUL ] commons-io#commons-io;2.4!commons-io.jar (23ms)
downloading https://repo1.maven.org/maven2/org/apache/commons/commons-lang3/3.5/commons-lang3-3.5.jar ...
    [SUCCESSFUL ] org.apache.commons#commons-lang3;3.5!commons-lang3.jar (33ms)
downloading https://repo1.maven.org/maven2/commons-lang/commons-lang/2.6/commons-lang-2.6.jar ...
    [SUCCESSFUL ] commons-lang#commons-lang;2.6!commons-lang.jar (24ms)
downloading https://repo1.maven.org/maven2/javolution/javolution/5.5.1/javolution-5.5.1.jar ...
    [SUCCESSFUL ] javolution#javolution;5.5.1!javolution.jar(bundle) (31ms)
downloading https://repo1.maven.org/maven2/log4j/apache-log4j-extras/1.2.17/apache-log4j-extras-1.2.17.jar ...
    [SUCCESSFUL ] log4j#apache-log4j-extras;1.2.17!apache-log4j-extras.jar(bundle) (33ms)
downloading https://repo1.maven.org/maven2/org/antlr/antlr-runtime/3.4/antlr-runtime-3.4.jar ...
    [SUCCESSFUL ] org.antlr#antlr-runtime;3.4!antlr-runtime.jar (19ms)
downloading https://repo1.maven.org/maven2/org/antlr/ST4/4.0.4/ST4-4.0.4.jar ...
    [SUCCESSFUL ] org.antlr#ST4;4.0.4!ST4.jar (25ms)
downloading https://repo1.maven.org/maven2/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar ...
    [SUCCESSFUL ] org.apache.commons#commons-compress;1.4.1!commons-compress.jar (23ms)
downloading https://repo1.maven.org/maven2/org/apache/ivy/ivy/2.4.0/ivy-2.4.0.jar ...
    [SUCCESSFUL ] org.apache.ivy#ivy;2.4.0!ivy.jar (69ms)
downloading https://repo1.maven.org/maven2/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar ...
    [SUCCESSFUL ] org.codehaus.jackson#jackson-core-asl;1.9.13!jackson-core-asl.jar (23ms)
downloading https://repo1.maven.org/maven2/com/googlecode/javaewah/JavaEWAH/0.3.2/JavaEWAH-0.3.2.jar ...
    [SUCCESSFUL ] com.googlecode.javaewah#JavaEWAH;0.3.2!JavaEWAH.jar (15ms)
downloading https://repo1.maven.org/maven2/org/iq80/snappy/snappy/0.2/snappy-0.2.jar ...
    [SUCCESSFUL ] org.iq80.snappy#snappy;0.2!snappy.jar (17ms)
downloading https://repo1.maven.org/maven2/stax/stax-api/1.0.1/stax-api-1.0.1.jar ...
    [SUCCESSFUL ] stax#stax-api;1.0.1!stax-api.jar (16ms)
downloading https://repo1.maven.org/maven2/net/sf/opencsv/opencsv/2.3/opencsv-2.3.jar ...
    [SUCCESSFUL ] net.sf.opencsv#opencsv;2.3!opencsv.jar (17ms)
downloading https://repo1.maven.org/maven2/org/antlr/stringtemplate/3.2.1/stringtemplate-3.2.1.jar ...
    [SUCCESSFUL ] org.antlr#stringtemplate;3.2.1!stringtemplate.jar (26ms)
downloading https://repo1.maven.org/maven2/antlr/antlr/2.7.7/antlr-2.7.7.jar ...
    [SUCCESSFUL ] antlr#antlr;2.7.7!antlr.jar (42ms)
downloading https://repo1.maven.org/maven2/com/thoughtworks/paranamer/paranamer/2.8/paranamer-2.8.jar ...
    [SUCCESSFUL ] com.thoughtworks.paranamer#paranamer;2.8!paranamer.jar(bundle) (16ms)
downloading https://repo1.maven.org/maven2/org/xerial/snappy/snappy-java/1.1.7.1/snappy-java-1.1.7.1.jar ...
    [SUCCESSFUL ] org.xerial.snappy#snappy-java;1.1.7.1!snappy-java.jar (97ms)
downloading https://repo1.maven.org/maven2/org/tukaani/xz/1.5/xz-1.5.jar ...
    [SUCCESSFUL ] org.tukaani#xz;1.5!xz.jar (18ms)
downloading https://repo1.maven.org/maven2/com/jolbox/bonecp/0.8.0.RELEASE/bonecp-0.8.0.RELEASE.jar ...
    [SUCCESSFUL ] com.jolbox#bonecp;0.8.0.RELEASE!bonecp.jar(bundle) (18ms)
downloading https://repo1.maven.org/maven2/commons-cli/commons-cli/1.2/commons-cli-1.2.jar ...
    [SUCCESSFUL ] commons-cli#commons-cli;1.2!commons-cli.jar (16ms)
downloading https://repo1.maven.org/maven2/org/datanucleus/datanucleus-api-jdo/3.2.6/datanucleus-api-jdo-3.2.6.jar ...
    [SUCCESSFUL ] org.datanucleus#datanucleus-api-jdo;3.2.6!datanucleus-api-jdo.jar (29ms)
downloading https://repo1.maven.org/maven2/org/datanucleus/datanucleus-rdbms/3.2.9/datanucleus-rdbms-3.2.9.jar ...
    [SUCCESSFUL ] org.datanucleus#datanucleus-rdbms;3.2.9!datanucleus-rdbms.jar (87ms)
downloading https://repo1.maven.org/maven2/commons-pool/commons-pool/1.5.4/commons-pool-1.5.4.jar ...
    [SUCCESSFUL ] commons-pool#commons-pool;1.5.4!commons-pool.jar (18ms)
downloading https://repo1.maven.org/maven2/commons-dbcp/commons-dbcp/1.4/commons-dbcp-1.4.jar ...
    [SUCCESSFUL ] commons-dbcp#commons-dbcp;1.4!commons-dbcp.jar (20ms)
downloading https://repo1.maven.org/maven2/javax/jdo/jdo-api/3.0.1/jdo-api-3.0.1.jar ...
    [SUCCESSFUL ] javax.jdo#jdo-api;3.0.1!jdo-api.jar (22ms)
downloading https://repo1.maven.org/maven2/javax/transaction/jta/1.1/jta-1.1.jar ...
    [SUCCESSFUL ] javax.transaction#jta;1.1!jta.jar (16ms)
downloading https://repo1.maven.org/maven2/org/apache/avro/avro-ipc/1.8.2/avro-ipc-1.8.2.jar ...
    [SUCCESSFUL ] org.apache.avro#avro-ipc;1.8.2!avro-ipc.jar(bundle) (19ms)
downloading https://repo1.maven.org/maven2/org/apache/calcite/calcite-linq4j/1.2.0-incubating/calcite-linq4j-1.2.0-incubating.jar ...
    [SUCCESSFUL ] org.apache.calcite#calcite-linq4j;1.2.0-incubating!calcite-linq4j.jar (35ms)
downloading https://repo1.maven.org/maven2/net/hydromatic/eigenbase-properties/1.1.5/eigenbase-properties-1.1.5.jar ...
    [SUCCESSFUL ] net.hydromatic#eigenbase-properties;1.1.5!eigenbase-properties.jar(bundle) (16ms)
downloading https://repo1.maven.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.jar ...
    [SUCCESSFUL ] commons-logging#commons-logging;1.2!commons-logging.jar (18ms)
downloading https://repo1.maven.org/maven2/com/typesafe/config/1.3.0/config-1.3.0.jar ...
    [SUCCESSFUL ] com.typesafe#config;1.3.0!config.jar(bundle) (26ms)
downloading https://repo1.maven.org/maven2/org/rocksdb/rocksdbjni/6.5.3/rocksdbjni-6.5.3.jar ...
    [SUCCESSFUL ] org.rocksdb#rocksdbjni;6.5.3!rocksdbjni.jar (1120ms)
downloading https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-aws/3.2.0/hadoop-aws-3.2.0.jar ...
    [SUCCESSFUL ] org.apache.hadoop#hadoop-aws;3.2.0!hadoop-aws.jar (30ms)
downloading https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.603/aws-java-sdk-core-1.11.603.jar ...
    [SUCCESSFUL ] com.amazonaws#aws-java-sdk-core;1.11.603!aws-java-sdk-core.jar (46ms)
downloading https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.603/aws-java-sdk-s3-1.11.603.jar ...
    [SUCCESSFUL ] com.amazonaws#aws-java-sdk-s3;1.11.603!aws-java-sdk-s3.jar (49ms)
downloading https://repo1.maven.org/maven2/com/github/universal-automata/liblevenshtein/3.0.0/liblevenshtein-3.0.0.jar ...
    [SUCCESSFUL ] com.github.universal-automata#liblevenshtein;3.0.0!liblevenshtein.jar (19ms)
downloading https://repo1.maven.org/maven2/com/navigamez/greex/1.0/greex-1.0.jar ...
    [SUCCESSFUL ] com.navigamez#greex;1.0!greex.jar (15ms)
downloading https://repo1.maven.org/maven2/org/json4s/json4s-ext_2.11/3.5.3/json4s-ext_2.11-3.5.3.jar ...
    [SUCCESSFUL ] org.json4s#json4s-ext_2.11;3.5.3!json4s-ext_2.11.jar (21ms)
downloading https://repo1.maven.org/maven2/org/tensorflow/tensorflow/1.15.0/tensorflow-1.15.0.jar ...
    [SUCCESSFUL ] org.tensorflow#tensorflow;1.15.0!tensorflow.jar (16ms)
downloading https://repo1.maven.org/maven2/net/sf/trove4j/trove4j/3.0.3/trove4j-3.0.3.jar ...
    [SUCCESSFUL ] net.sf.trove4j#trove4j;3.0.3!trove4j.jar (109ms)
downloading https://repo1.maven.org/maven2/org/apache/httpcomponents/httpclient/4.5.9/httpclient-4.5.9.jar ...
    [SUCCESSFUL ] org.apache.httpcomponents#httpclient;4.5.9!httpclient.jar (39ms)
downloading https://repo1.maven.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.jar ...
    [SUCCESSFUL ] software.amazon.ion#ion-java;1.0.2!ion-java.jar(bundle) (33ms)
downloading https://repo1.maven.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.jar ...
    [SUCCESSFUL ] com.fasterxml.jackson.dataformat#jackson-dataformat-cbor;2.6.7!jackson-dataformat-cbor.jar(bundle) (17ms)
downloading https://repo1.maven.org/maven2/org/apache/httpcomponents/httpcore/4.4.11/httpcore-4.4.11.jar ...
    [SUCCESSFUL ] org.apache.httpcomponents#httpcore;4.4.11!httpcore.jar (45ms)
downloading https://repo1.maven.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.jar ...
    [SUCCESSFUL ] commons-codec#commons-codec;1.11!commons-codec.jar (29ms)
downloading https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.603/aws-java-sdk-kms-1.11.603.jar ...
    [SUCCESSFUL ] com.amazonaws#aws-java-sdk-kms;1.11.603!aws-java-sdk-kms.jar (32ms)
downloading https://repo1.maven.org/maven2/com/amazonaws/jmespath-java/1.11.603/jmespath-java-1.11.603.jar ...
    [SUCCESSFUL ] com.amazonaws#jmespath-java;1.11.603!jmespath-java.jar (16ms)
downloading https://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.2/jackson-databind-2.6.7.2.jar ...
    [SUCCESSFUL ] com.fasterxml.jackson.core#jackson-databind;2.6.7.2!jackson-databind.jar(bundle) (55ms)
downloading https://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.jar ...
    [SUCCESSFUL ] com.fasterxml.jackson.core#jackson-annotations;2.6.0!jackson-annotations.jar(bundle) (16ms)
downloading https://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.jar ...
    [SUCCESSFUL ] com.fasterxml.jackson.core#jackson-core;2.6.7!jackson-core.jar(bundle) (22ms)
downloading https://repo1.maven.org/maven2/com/google/code/findbugs/annotations/3.0.1/annotations-3.0.1.jar ...
    [SUCCESSFUL ] com.google.code.findbugs#annotations;3.0.1!annotations.jar (16ms)
downloading https://repo1.maven.org/maven2/com/google/protobuf/protobuf-java-util/3.0.0-beta-3/protobuf-java-util-3.0.0-beta-3.jar ...
    [SUCCESSFUL ] com.google.protobuf#protobuf-java-util;3.0.0-beta-3!protobuf-java-util.jar(bundle) (16ms)
downloading https://repo1.maven.org/maven2/com/google/protobuf/protobuf-java/3.0.0-beta-3/protobuf-java-3.0.0-beta-3.jar ...
    [SUCCESSFUL ] com.google.protobuf#protobuf-java;3.0.0-beta-3!protobuf-java.jar(bundle) (53ms)
downloading https://repo1.maven.org/maven2/it/unimi/dsi/fastutil/7.0.12/fastutil-7.0.12.jar ...
    [SUCCESSFUL ] it.unimi.dsi#fastutil;7.0.12!fastutil.jar (592ms)
downloading https://repo1.maven.org/maven2/org/projectlombok/lombok/1.16.8/lombok-1.16.8.jar ...
    [SUCCESSFUL ] org.projectlombok#lombok;1.16.8!lombok.jar (57ms)
downloading https://repo1.maven.org/maven2/org/slf4j/slf4j-api/1.7.21/slf4j-api-1.7.21.jar ...
    [SUCCESSFUL ] org.slf4j#slf4j-api;1.7.21!slf4j-api.jar (17ms)
downloading https://repo1.maven.org/maven2/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.jar ...
    [SUCCESSFUL ] net.jcip#jcip-annotations;1.0!jcip-annotations.jar (14ms)
downloading https://repo1.maven.org/maven2/com/google/code/findbugs/jsr305/3.0.1/jsr305-3.0.1.jar ...
    [SUCCESSFUL ] com.google.code.findbugs#jsr305;3.0.1!jsr305.jar (15ms)
downloading https://repo1.maven.org/maven2/com/google/code/gson/gson/2.3/gson-2.3.jar ...
    [SUCCESSFUL ] com.google.code.gson#gson;2.3!gson.jar (23ms)
downloading https://repo1.maven.org/maven2/dk/brics/automaton/automaton/1.11-8/automaton-1.11-8.jar ...
    [SUCCESSFUL ] dk.brics.automaton#automaton;1.11-8!automaton.jar (20ms)
downloading https://repo1.maven.org/maven2/joda-time/joda-time/2.9.5/joda-time-2.9.5.jar ...
    [SUCCESSFUL ] joda-time#joda-time;2.9.5!joda-time.jar (34ms)
downloading https://repo1.maven.org/maven2/org/joda/joda-convert/1.8.1/joda-convert-1.8.1.jar ...
    [SUCCESSFUL ] org.joda#joda-convert;1.8.1!joda-convert.jar (18ms)
downloading https://repo1.maven.org/maven2/org/tensorflow/libtensorflow/1.15.0/libtensorflow-1.15.0.jar ...
    [SUCCESSFUL ] org.tensorflow#libtensorflow;1.15.0!libtensorflow.jar (78ms)
downloading https://repo1.maven.org/maven2/org/tensorflow/libtensorflow_jni/1.15.0/libtensorflow_jni-1.15.0.jar ...
    [SUCCESSFUL ] org.tensorflow#libtensorflow_jni;1.15.0!libtensorflow_jni.jar (5308ms)
:: resolution report :: resolve 16070ms :: artifacts dl 10952ms
    :: modules in use:
    antlr#antlr;2.7.7 from central in [default]
    com.amazonaws#aws-java-sdk-core;1.11.603 from central in [default]
    com.amazonaws#aws-java-sdk-kms;1.11.603 from central in [default]
    com.amazonaws#aws-java-sdk-s3;1.11.603 from central in [default]
    com.amazonaws#jmespath-java;1.11.603 from central in [default]
    com.fasterxml.jackson.core#jackson-annotations;2.6.0 from central in [default]
    com.fasterxml.jackson.core#jackson-core;2.6.7 from central in [default]
    com.fasterxml.jackson.core#jackson-databind;2.6.7.2 from central in [default]
    com.fasterxml.jackson.dataformat#jackson-dataformat-cbor;2.6.7 from central in [default]
    com.github.universal-automata#liblevenshtein;3.0.0 from central in [default]
    com.google.code.findbugs#annotations;3.0.1 from central in [default]
    com.google.code.findbugs#jsr305;3.0.1 from central in [default]
    com.google.code.gson#gson;2.3 from central in [default]
    com.google.protobuf#protobuf-java;3.0.0-beta-3 from central in [default]
    com.google.protobuf#protobuf-java-util;3.0.0-beta-3 from central in [default]
    com.googlecode.javaewah#JavaEWAH;0.3.2 from central in [default]
    com.johnsnowlabs.nlp#spark-nlp_2.11;2.5.4 from central in [default]
    com.jolbox#bonecp;0.8.0.RELEASE from central in [default]
    com.navigamez#greex;1.0 from central in [default]
    com.thoughtworks.paranamer#paranamer;2.8 from central in [default]
    com.twitter#parquet-hadoop-bundle;1.6.0 from central in [default]
    com.typesafe#config;1.3.0 from central in [default]
    commons-cli#commons-cli;1.2 from central in [default]
    commons-codec#commons-codec;1.11 from central in [default]
    commons-dbcp#commons-dbcp;1.4 from central in [default]
    commons-httpclient#commons-httpclient;3.1 from central in [default]
    commons-io#commons-io;2.4 from central in [default]
    commons-lang#commons-lang;2.6 from central in [default]
    commons-logging#commons-logging;1.2 from central in [default]
    commons-pool#commons-pool;1.5.4 from central in [default]
    dk.brics.automaton#automaton;1.11-8 from central in [default]
    it.unimi.dsi#fastutil;7.0.12 from central in [default]
    javax.jdo#jdo-api;3.0.1 from central in [default]
    javax.transaction#jta;1.1 from central in [default]
    javolution#javolution;5.5.1 from central in [default]
    joda-time#joda-time;2.9.5 from central in [default]
    log4j#apache-log4j-extras;1.2.17 from central in [default]
    net.hydromatic#eigenbase-properties;1.1.5 from central in [default]
    net.jcip#jcip-annotations;1.0 from central in [default]
    net.sf.opencsv#opencsv;2.3 from central in [default]
    net.sf.trove4j#trove4j;3.0.3 from central in [default]
    org.antlr#ST4;4.0.4 from central in [default]
    org.antlr#antlr-runtime;3.4 from central in [default]
    org.antlr#stringtemplate;3.2.1 from central in [default]
    org.apache.avro#avro;1.8.2 from central in [default]
    org.apache.avro#avro-ipc;1.8.2 from central in [default]
    org.apache.avro#avro-mapred;1.8.2 from central in [default]
    org.apache.calcite#calcite-avatica;1.2.0-incubating from central in [default]
    org.apache.calcite#calcite-core;1.2.0-incubating from central in [default]
    org.apache.calcite#calcite-linq4j;1.2.0-incubating from central in [default]
    org.apache.commons#commons-compress;1.4.1 from central in [default]
    org.apache.commons#commons-lang3;3.5 from central in [default]
    org.apache.derby#derby;10.12.1.1 from central in [default]
    org.apache.hadoop#hadoop-aws;3.2.0 from central in [default]
    org.apache.httpcomponents#httpclient;4.5.9 from central in [default]
    org.apache.httpcomponents#httpcore;4.4.11 from central in [default]
    org.apache.ivy#ivy;2.4.0 from central in [default]
    org.apache.spark#spark-hive_2.11;2.4.0 from central in [default]
    org.apache.thrift#libfb303;0.9.3 from central in [default]
    org.apache.thrift#libthrift;0.9.3 from central in [default]
    org.codehaus.jackson#jackson-core-asl;1.9.13 from central in [default]
    org.codehaus.jackson#jackson-mapper-asl;1.9.13 from central in [default]
    org.datanucleus#datanucleus-api-jdo;3.2.6 from central in [default]
    org.datanucleus#datanucleus-core;3.2.10 from central in [default]
    org.datanucleus#datanucleus-rdbms;3.2.9 from central in [default]
    org.iq80.snappy#snappy;0.2 from central in [default]
    org.joda#joda-convert;1.8.1 from central in [default]
    org.jodd#jodd-core;3.5.2 from central in [default]
    org.json4s#json4s-ext_2.11;3.5.3 from central in [default]
    org.projectlombok#lombok;1.16.8 from central in [default]
    org.rocksdb#rocksdbjni;6.5.3 from central in [default]
    org.slf4j#slf4j-api;1.7.21 from central in [default]
    org.spark-project.hive#hive-exec;1.2.1.spark2 from central in [default]
    org.spark-project.hive#hive-metastore;1.2.1.spark2 from central in [default]
    org.spark-project.spark#unused;1.0.0 from central in [default]
    org.tensorflow#libtensorflow;1.15.0 from central in [default]
    org.tensorflow#libtensorflow_jni;1.15.0 from central in [default]
    org.tensorflow#tensorflow;1.15.0 from central in [default]
    org.tukaani#xz;1.5 from central in [default]
    org.xerial.snappy#snappy-java;1.1.7.1 from central in [default]
    software.amazon.ion#ion-java;1.0.2 from central in [default]
    stax#stax-api;1.0.1 from central in [default]
    :: evicted modules:
    org.tukaani#xz;1.0 by [org.tukaani#xz;1.5] in [default]
    org.slf4j#slf4j-api;1.7.16 by [org.slf4j#slf4j-api;1.7.21] in [default]
    commons-logging#commons-logging;1.1.3 by [commons-logging#commons-logging;1.2] in [default]
    commons-codec#commons-codec;1.10 by [commons-codec#commons-codec;1.11] in [default]
    commons-logging#commons-logging;1.0.4 by [commons-logging#commons-logging;1.1.3] in [default]
    org.apache.httpcomponents#httpclient;4.5.6 by [org.apache.httpcomponents#httpclient;4.5.9] in [default]
    org.apache.httpcomponents#httpcore;4.4.10 by [org.apache.httpcomponents#httpcore;4.4.11] in [default]
    joda-time#joda-time;2.9.3 by [joda-time#joda-time;2.9.5] in [default]
    com.google.code.findbugs#jsr305;1.3.9 by [com.google.code.findbugs#jsr305;3.0.1] in [default]
    joda-time#joda-time;2.8.1 by [joda-time#joda-time;2.9.5] in [default]
    ---------------------------------------------------------------------
    |                  |            modules            ||   artifacts   |
    |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
    ---------------------------------------------------------------------
    |      default     |   92  |   89  |   89  |   10  ||   82  |   82  |
    ---------------------------------------------------------------------

:: problems summary ::
:::: ERRORS
    unknown resolver null

    unknown resolver null

    unknown resolver null

    unknown resolver redefined-public

    unknown resolver null

    unknown resolver null

    unknown resolver null

    unknown resolver null

    unknown resolver null

    unknown resolver null

    unknown resolver null

    unknown resolver null

    unknown resolver null

    unknown resolver null

    unknown resolver null

    unknown resolver null

    unknown resolver null

    unknown resolver null

    unknown resolver null

    unknown resolver null

:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
:: retrieving :: org.apache.spark#spark-submit-parent-8c906343-c23f-41b7-ab1d-64fd3eccb6f2
    confs: [default]
    82 artifacts copied, 0 already retrieved (269355kB/280ms)
20/08/06 13:50:06 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
20/08/06 13:50:07 WARN DependencyUtils: Local jar /root/.ivy2/jars/org.apache.avro_avro-mapred-1.8.2.jar does not exist, skipping.
20/08/06 13:50:07 INFO SparkContext: Running Spark version 2.4.3
20/08/06 13:50:07 INFO SparkContext: Submitted application: spark-nlp-starter
20/08/06 13:50:07 INFO SecurityManager: Changing view acls to: root
20/08/06 13:50:07 INFO SecurityManager: Changing modify acls to: root
20/08/06 13:50:07 INFO SecurityManager: Changing view acls groups to: 
20/08/06 13:50:07 INFO SecurityManager: Changing modify acls groups to: 
20/08/06 13:50:07 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
20/08/06 13:50:07 INFO Utils: Successfully started service 'sparkDriver' on port 38823.
20/08/06 13:50:07 INFO SparkEnv: Registering MapOutputTracker
20/08/06 13:50:07 INFO SparkEnv: Registering BlockManagerMaster
20/08/06 13:50:07 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
20/08/06 13:50:07 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
20/08/06 13:50:07 INFO DiskBlockManager: Created local directory at /tmp/blockmgr-a0713ff5-b8f2-424d-a168-b497acd8a6fa
20/08/06 13:50:07 INFO MemoryStore: MemoryStore started with capacity 2004.6 MB
20/08/06 13:50:07 INFO SparkEnv: Registering OutputCommitCoordinator
20/08/06 13:50:08 INFO Utils: Successfully started service 'SparkUI' on port 4040.
20/08/06 13:50:08 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://56f1369e0bb4:4040
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.apache.spark_spark-hive_2.11-2.4.0.jar at spark://56f1369e0bb4:38823/jars/org.apache.spark_spark-hive_2.11-2.4.0.jar with timestamp 1596721808126
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.johnsnowlabs.nlp_spark-nlp_2.11-2.5.4.jar at spark://56f1369e0bb4:38823/jars/com.johnsnowlabs.nlp_spark-nlp_2.11-2.5.4.jar with timestamp 1596721808127
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.twitter_parquet-hadoop-bundle-1.6.0.jar at spark://56f1369e0bb4:38823/jars/com.twitter_parquet-hadoop-bundle-1.6.0.jar with timestamp 1596721808128
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.spark-project.hive_hive-exec-1.2.1.spark2.jar at spark://56f1369e0bb4:38823/jars/org.spark-project.hive_hive-exec-1.2.1.spark2.jar with timestamp 1596721808128
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.spark-project.hive_hive-metastore-1.2.1.spark2.jar at spark://56f1369e0bb4:38823/jars/org.spark-project.hive_hive-metastore-1.2.1.spark2.jar with timestamp 1596721808128
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.apache.avro_avro-1.8.2.jar at spark://56f1369e0bb4:38823/jars/org.apache.avro_avro-1.8.2.jar with timestamp 1596721808128
20/08/06 13:50:08 ERROR SparkContext: Failed to add file:/root/.ivy2/jars/org.apache.avro_avro-mapred-1.8.2.jar to Spark environment
java.io.FileNotFoundException: Jar /root/.ivy2/jars/org.apache.avro_avro-mapred-1.8.2.jar not found
    at org.apache.spark.SparkContext.addJarFile$1(SparkContext.scala:1838)
    at org.apache.spark.SparkContext.addJar(SparkContext.scala:1868)
    at org.apache.spark.SparkContext$$anonfun$12.apply(SparkContext.scala:458)
    at org.apache.spark.SparkContext$$anonfun$12.apply(SparkContext.scala:458)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:458)
    at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2520)
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:935)
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:926)
    at scala.Option.getOrElse(Option.scala:121)
    at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:926)
    at Main$.main(Main.scala:13)
    at Main.main(Main.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
    at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
    at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
    at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
    at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
    at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/commons-httpclient_commons-httpclient-3.1.jar at spark://56f1369e0bb4:38823/jars/commons-httpclient_commons-httpclient-3.1.jar with timestamp 1596721808132
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.apache.calcite_calcite-avatica-1.2.0-incubating.jar at spark://56f1369e0bb4:38823/jars/org.apache.calcite_calcite-avatica-1.2.0-incubating.jar with timestamp 1596721808132
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.apache.calcite_calcite-core-1.2.0-incubating.jar at spark://56f1369e0bb4:38823/jars/org.apache.calcite_calcite-core-1.2.0-incubating.jar with timestamp 1596721808132
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.codehaus.jackson_jackson-mapper-asl-1.9.13.jar at spark://56f1369e0bb4:38823/jars/org.codehaus.jackson_jackson-mapper-asl-1.9.13.jar with timestamp 1596721808132
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.jodd_jodd-core-3.5.2.jar at spark://56f1369e0bb4:38823/jars/org.jodd_jodd-core-3.5.2.jar with timestamp 1596721808132
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.datanucleus_datanucleus-core-3.2.10.jar at spark://56f1369e0bb4:38823/jars/org.datanucleus_datanucleus-core-3.2.10.jar with timestamp 1596721808132
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.apache.thrift_libthrift-0.9.3.jar at spark://56f1369e0bb4:38823/jars/org.apache.thrift_libthrift-0.9.3.jar with timestamp 1596721808132
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.apache.thrift_libfb303-0.9.3.jar at spark://56f1369e0bb4:38823/jars/org.apache.thrift_libfb303-0.9.3.jar with timestamp 1596721808132
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.apache.derby_derby-10.12.1.1.jar at spark://56f1369e0bb4:38823/jars/org.apache.derby_derby-10.12.1.1.jar with timestamp 1596721808132
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.spark-project.spark_unused-1.0.0.jar at spark://56f1369e0bb4:38823/jars/org.spark-project.spark_unused-1.0.0.jar with timestamp 1596721808133
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/commons-io_commons-io-2.4.jar at spark://56f1369e0bb4:38823/jars/commons-io_commons-io-2.4.jar with timestamp 1596721808133
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.apache.commons_commons-lang3-3.5.jar at spark://56f1369e0bb4:38823/jars/org.apache.commons_commons-lang3-3.5.jar with timestamp 1596721808133
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/commons-lang_commons-lang-2.6.jar at spark://56f1369e0bb4:38823/jars/commons-lang_commons-lang-2.6.jar with timestamp 1596721808133
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/javolution_javolution-5.5.1.jar at spark://56f1369e0bb4:38823/jars/javolution_javolution-5.5.1.jar with timestamp 1596721808133
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/log4j_apache-log4j-extras-1.2.17.jar at spark://56f1369e0bb4:38823/jars/log4j_apache-log4j-extras-1.2.17.jar with timestamp 1596721808133
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.antlr_antlr-runtime-3.4.jar at spark://56f1369e0bb4:38823/jars/org.antlr_antlr-runtime-3.4.jar with timestamp 1596721808133
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.antlr_ST4-4.0.4.jar at spark://56f1369e0bb4:38823/jars/org.antlr_ST4-4.0.4.jar with timestamp 1596721808133
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.apache.commons_commons-compress-1.4.1.jar at spark://56f1369e0bb4:38823/jars/org.apache.commons_commons-compress-1.4.1.jar with timestamp 1596721808133
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.apache.ivy_ivy-2.4.0.jar at spark://56f1369e0bb4:38823/jars/org.apache.ivy_ivy-2.4.0.jar with timestamp 1596721808133
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.codehaus.jackson_jackson-core-asl-1.9.13.jar at spark://56f1369e0bb4:38823/jars/org.codehaus.jackson_jackson-core-asl-1.9.13.jar with timestamp 1596721808133
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.googlecode.javaewah_JavaEWAH-0.3.2.jar at spark://56f1369e0bb4:38823/jars/com.googlecode.javaewah_JavaEWAH-0.3.2.jar with timestamp 1596721808133
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.iq80.snappy_snappy-0.2.jar at spark://56f1369e0bb4:38823/jars/org.iq80.snappy_snappy-0.2.jar with timestamp 1596721808133
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/stax_stax-api-1.0.1.jar at spark://56f1369e0bb4:38823/jars/stax_stax-api-1.0.1.jar with timestamp 1596721808133
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/net.sf.opencsv_opencsv-2.3.jar at spark://56f1369e0bb4:38823/jars/net.sf.opencsv_opencsv-2.3.jar with timestamp 1596721808134
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.antlr_stringtemplate-3.2.1.jar at spark://56f1369e0bb4:38823/jars/org.antlr_stringtemplate-3.2.1.jar with timestamp 1596721808134
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/antlr_antlr-2.7.7.jar at spark://56f1369e0bb4:38823/jars/antlr_antlr-2.7.7.jar with timestamp 1596721808134
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.thoughtworks.paranamer_paranamer-2.8.jar at spark://56f1369e0bb4:38823/jars/com.thoughtworks.paranamer_paranamer-2.8.jar with timestamp 1596721808134
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.xerial.snappy_snappy-java-1.1.7.1.jar at spark://56f1369e0bb4:38823/jars/org.xerial.snappy_snappy-java-1.1.7.1.jar with timestamp 1596721808134
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.tukaani_xz-1.5.jar at spark://56f1369e0bb4:38823/jars/org.tukaani_xz-1.5.jar with timestamp 1596721808134
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.jolbox_bonecp-0.8.0.RELEASE.jar at spark://56f1369e0bb4:38823/jars/com.jolbox_bonecp-0.8.0.RELEASE.jar with timestamp 1596721808134
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/commons-cli_commons-cli-1.2.jar at spark://56f1369e0bb4:38823/jars/commons-cli_commons-cli-1.2.jar with timestamp 1596721808134
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.datanucleus_datanucleus-api-jdo-3.2.6.jar at spark://56f1369e0bb4:38823/jars/org.datanucleus_datanucleus-api-jdo-3.2.6.jar with timestamp 1596721808134
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.datanucleus_datanucleus-rdbms-3.2.9.jar at spark://56f1369e0bb4:38823/jars/org.datanucleus_datanucleus-rdbms-3.2.9.jar with timestamp 1596721808139
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/commons-pool_commons-pool-1.5.4.jar at spark://56f1369e0bb4:38823/jars/commons-pool_commons-pool-1.5.4.jar with timestamp 1596721808139
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/commons-dbcp_commons-dbcp-1.4.jar at spark://56f1369e0bb4:38823/jars/commons-dbcp_commons-dbcp-1.4.jar with timestamp 1596721808140
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/javax.jdo_jdo-api-3.0.1.jar at spark://56f1369e0bb4:38823/jars/javax.jdo_jdo-api-3.0.1.jar with timestamp 1596721808140
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/javax.transaction_jta-1.1.jar at spark://56f1369e0bb4:38823/jars/javax.transaction_jta-1.1.jar with timestamp 1596721808141
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.apache.avro_avro-ipc-1.8.2.jar at spark://56f1369e0bb4:38823/jars/org.apache.avro_avro-ipc-1.8.2.jar with timestamp 1596721808141
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.apache.calcite_calcite-linq4j-1.2.0-incubating.jar at spark://56f1369e0bb4:38823/jars/org.apache.calcite_calcite-linq4j-1.2.0-incubating.jar with timestamp 1596721808142
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/net.hydromatic_eigenbase-properties-1.1.5.jar at spark://56f1369e0bb4:38823/jars/net.hydromatic_eigenbase-properties-1.1.5.jar with timestamp 1596721808142
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/commons-logging_commons-logging-1.2.jar at spark://56f1369e0bb4:38823/jars/commons-logging_commons-logging-1.2.jar with timestamp 1596721808142
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.typesafe_config-1.3.0.jar at spark://56f1369e0bb4:38823/jars/com.typesafe_config-1.3.0.jar with timestamp 1596721808142
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.rocksdb_rocksdbjni-6.5.3.jar at spark://56f1369e0bb4:38823/jars/org.rocksdb_rocksdbjni-6.5.3.jar with timestamp 1596721808143
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.apache.hadoop_hadoop-aws-3.2.0.jar at spark://56f1369e0bb4:38823/jars/org.apache.hadoop_hadoop-aws-3.2.0.jar with timestamp 1596721808143
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.amazonaws_aws-java-sdk-core-1.11.603.jar at spark://56f1369e0bb4:38823/jars/com.amazonaws_aws-java-sdk-core-1.11.603.jar with timestamp 1596721808143
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.amazonaws_aws-java-sdk-s3-1.11.603.jar at spark://56f1369e0bb4:38823/jars/com.amazonaws_aws-java-sdk-s3-1.11.603.jar with timestamp 1596721808143
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.github.universal-automata_liblevenshtein-3.0.0.jar at spark://56f1369e0bb4:38823/jars/com.github.universal-automata_liblevenshtein-3.0.0.jar with timestamp 1596721808143
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.navigamez_greex-1.0.jar at spark://56f1369e0bb4:38823/jars/com.navigamez_greex-1.0.jar with timestamp 1596721808144
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.json4s_json4s-ext_2.11-3.5.3.jar at spark://56f1369e0bb4:38823/jars/org.json4s_json4s-ext_2.11-3.5.3.jar with timestamp 1596721808144
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.tensorflow_tensorflow-1.15.0.jar at spark://56f1369e0bb4:38823/jars/org.tensorflow_tensorflow-1.15.0.jar with timestamp 1596721808144
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/net.sf.trove4j_trove4j-3.0.3.jar at spark://56f1369e0bb4:38823/jars/net.sf.trove4j_trove4j-3.0.3.jar with timestamp 1596721808144
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.apache.httpcomponents_httpclient-4.5.9.jar at spark://56f1369e0bb4:38823/jars/org.apache.httpcomponents_httpclient-4.5.9.jar with timestamp 1596721808145
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/software.amazon.ion_ion-java-1.0.2.jar at spark://56f1369e0bb4:38823/jars/software.amazon.ion_ion-java-1.0.2.jar with timestamp 1596721808145
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.fasterxml.jackson.dataformat_jackson-dataformat-cbor-2.6.7.jar at spark://56f1369e0bb4:38823/jars/com.fasterxml.jackson.dataformat_jackson-dataformat-cbor-2.6.7.jar with timestamp 1596721808145
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.apache.httpcomponents_httpcore-4.4.11.jar at spark://56f1369e0bb4:38823/jars/org.apache.httpcomponents_httpcore-4.4.11.jar with timestamp 1596721808146
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/commons-codec_commons-codec-1.11.jar at spark://56f1369e0bb4:38823/jars/commons-codec_commons-codec-1.11.jar with timestamp 1596721808146
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.amazonaws_aws-java-sdk-kms-1.11.603.jar at spark://56f1369e0bb4:38823/jars/com.amazonaws_aws-java-sdk-kms-1.11.603.jar with timestamp 1596721808146
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.amazonaws_jmespath-java-1.11.603.jar at spark://56f1369e0bb4:38823/jars/com.amazonaws_jmespath-java-1.11.603.jar with timestamp 1596721808147
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.fasterxml.jackson.core_jackson-databind-2.6.7.2.jar at spark://56f1369e0bb4:38823/jars/com.fasterxml.jackson.core_jackson-databind-2.6.7.2.jar with timestamp 1596721808147
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.fasterxml.jackson.core_jackson-annotations-2.6.0.jar at spark://56f1369e0bb4:38823/jars/com.fasterxml.jackson.core_jackson-annotations-2.6.0.jar with timestamp 1596721808147
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.fasterxml.jackson.core_jackson-core-2.6.7.jar at spark://56f1369e0bb4:38823/jars/com.fasterxml.jackson.core_jackson-core-2.6.7.jar with timestamp 1596721808147
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.google.code.findbugs_annotations-3.0.1.jar at spark://56f1369e0bb4:38823/jars/com.google.code.findbugs_annotations-3.0.1.jar with timestamp 1596721808147
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.google.protobuf_protobuf-java-util-3.0.0-beta-3.jar at spark://56f1369e0bb4:38823/jars/com.google.protobuf_protobuf-java-util-3.0.0-beta-3.jar with timestamp 1596721808148
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.google.protobuf_protobuf-java-3.0.0-beta-3.jar at spark://56f1369e0bb4:38823/jars/com.google.protobuf_protobuf-java-3.0.0-beta-3.jar with timestamp 1596721808148
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/it.unimi.dsi_fastutil-7.0.12.jar at spark://56f1369e0bb4:38823/jars/it.unimi.dsi_fastutil-7.0.12.jar with timestamp 1596721808149
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.projectlombok_lombok-1.16.8.jar at spark://56f1369e0bb4:38823/jars/org.projectlombok_lombok-1.16.8.jar with timestamp 1596721808149
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.slf4j_slf4j-api-1.7.21.jar at spark://56f1369e0bb4:38823/jars/org.slf4j_slf4j-api-1.7.21.jar with timestamp 1596721808150
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/net.jcip_jcip-annotations-1.0.jar at spark://56f1369e0bb4:38823/jars/net.jcip_jcip-annotations-1.0.jar with timestamp 1596721808150
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.google.code.findbugs_jsr305-3.0.1.jar at spark://56f1369e0bb4:38823/jars/com.google.code.findbugs_jsr305-3.0.1.jar with timestamp 1596721808150
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/com.google.code.gson_gson-2.3.jar at spark://56f1369e0bb4:38823/jars/com.google.code.gson_gson-2.3.jar with timestamp 1596721808151
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/dk.brics.automaton_automaton-1.11-8.jar at spark://56f1369e0bb4:38823/jars/dk.brics.automaton_automaton-1.11-8.jar with timestamp 1596721808151
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/joda-time_joda-time-2.9.5.jar at spark://56f1369e0bb4:38823/jars/joda-time_joda-time-2.9.5.jar with timestamp 1596721808151
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.joda_joda-convert-1.8.1.jar at spark://56f1369e0bb4:38823/jars/org.joda_joda-convert-1.8.1.jar with timestamp 1596721808151
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.tensorflow_libtensorflow-1.15.0.jar at spark://56f1369e0bb4:38823/jars/org.tensorflow_libtensorflow-1.15.0.jar with timestamp 1596721808151
20/08/06 13:50:08 INFO SparkContext: Added JAR file:///root/.ivy2/jars/org.tensorflow_libtensorflow_jni-1.15.0.jar at spark://56f1369e0bb4:38823/jars/org.tensorflow_libtensorflow_jni-1.15.0.jar with timestamp 1596721808151
20/08/06 13:50:08 INFO SparkContext: Added JAR file:/deploy/spark-nlp-starter-fix.jar at spark://56f1369e0bb4:38823/jars/spark-nlp-starter-fix.jar with timestamp 1596721808151
20/08/06 13:50:08 INFO Executor: Starting executor ID driver on host localhost
20/08/06 13:50:08 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 43539.
20/08/06 13:50:08 INFO NettyBlockTransferService: Server created on 56f1369e0bb4:43539
20/08/06 13:50:08 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
20/08/06 13:50:08 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 56f1369e0bb4, 43539, None)
20/08/06 13:50:08 INFO BlockManagerMasterEndpoint: Registering block manager 56f1369e0bb4:43539 with 2004.6 MB RAM, BlockManagerId(driver, 56f1369e0bb4, 43539, None)
20/08/06 13:50:08 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 56f1369e0bb4, 43539, None)
20/08/06 13:50:08 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, 56f1369e0bb4, 43539, None)
glove_100d download started this may take some time.
20/08/06 13:50:09 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
20/08/06 13:50:09 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
20/08/06 13:50:09 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
20/08/06 13:50:09 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
20/08/06 13:50:10 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
20/08/06 13:50:10 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
20/08/06 13:50:10 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
20/08/06 13:50:10 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
Approximate size to download 145.3 MB
20/08/06 13:50:10 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
20/08/06 13:50:10 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
Download done! Loading the resource.
20/08/06 13:50:25 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 236.7 KB, free 2004.4 MB)
20/08/06 13:50:25 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 22.9 KB, free 2004.3 MB)
20/08/06 13:50:25 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 56f1369e0bb4:43539 (size: 22.9 KB, free: 2004.6 MB)
20/08/06 13:50:25 INFO SparkContext: Created broadcast 0 from textFile at ReadWrite.scala:615
20/08/06 13:50:25 INFO FileInputFormat: Total input paths to process : 1
20/08/06 13:50:25 INFO SparkContext: Starting job: first at ReadWrite.scala:615
20/08/06 13:50:25 INFO DAGScheduler: Got job 0 (first at ReadWrite.scala:615) with 1 output partitions
20/08/06 13:50:25 INFO DAGScheduler: Final stage: ResultStage 0 (first at ReadWrite.scala:615)
20/08/06 13:50:25 INFO DAGScheduler: Parents of final stage: List()
20/08/06 13:50:25 INFO DAGScheduler: Missing parents: List()
20/08/06 13:50:25 INFO DAGScheduler: Submitting ResultStage 0 (file:/root/cache_pretrained/glove_100d_en_2.4.0_2.4_1579690104032/metadata MapPartitionsRDD[1] at textFile at ReadWrite.scala:615), which has no missing parents
20/08/06 13:50:25 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 3.5 KB, free 2004.3 MB)
20/08/06 13:50:25 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 2.1 KB, free 2004.3 MB)
20/08/06 13:50:25 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on 56f1369e0bb4:43539 (size: 2.1 KB, free: 2004.6 MB)
20/08/06 13:50:25 INFO SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:1161
20/08/06 13:50:25 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 0 (file:/root/cache_pretrained/glove_100d_en_2.4.0_2.4_1579690104032/metadata MapPartitionsRDD[1] at textFile at ReadWrite.scala:615) (first 15 tasks are for partitions Vector(0))
20/08/06 13:50:25 INFO TaskSchedulerImpl: Adding task set 0.0 with 1 tasks
20/08/06 13:50:25 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, localhost, executor driver, partition 0, PROCESS_LOCAL, 7945 bytes)
20/08/06 13:50:25 INFO Executor: Running task 0.0 in stage 0.0 (TID 0)
20/08/06 13:50:25 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/javolution_javolution-5.5.1.jar with timestamp 1596721808133
20/08/06 13:50:25 INFO TransportClientFactory: Successfully created connection to 56f1369e0bb4/172.19.0.4:38823 after 31 ms (0 ms spent in bootstraps)
20/08/06 13:50:25 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/javolution_javolution-5.5.1.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp6883256981348951494.tmp
20/08/06 13:50:25 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/javolution_javolution-5.5.1.jar to class loader
20/08/06 13:50:25 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.amazonaws_aws-java-sdk-s3-1.11.603.jar with timestamp 1596721808143
20/08/06 13:50:25 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.amazonaws_aws-java-sdk-s3-1.11.603.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp7336357957725497075.tmp
20/08/06 13:50:25 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.amazonaws_aws-java-sdk-s3-1.11.603.jar to class loader
20/08/06 13:50:25 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/net.sf.opencsv_opencsv-2.3.jar with timestamp 1596721808134
20/08/06 13:50:25 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/net.sf.opencsv_opencsv-2.3.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp7954066886674572874.tmp
20/08/06 13:50:25 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/net.sf.opencsv_opencsv-2.3.jar to class loader
20/08/06 13:50:25 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/software.amazon.ion_ion-java-1.0.2.jar with timestamp 1596721808145
20/08/06 13:50:25 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/software.amazon.ion_ion-java-1.0.2.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp7062465146580968433.tmp
20/08/06 13:50:25 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/software.amazon.ion_ion-java-1.0.2.jar to class loader
20/08/06 13:50:25 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.googlecode.javaewah_JavaEWAH-0.3.2.jar with timestamp 1596721808133
20/08/06 13:50:25 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.googlecode.javaewah_JavaEWAH-0.3.2.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp771747964091584281.tmp
20/08/06 13:50:25 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.googlecode.javaewah_JavaEWAH-0.3.2.jar to class loader
20/08/06 13:50:25 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/commons-codec_commons-codec-1.11.jar with timestamp 1596721808146
20/08/06 13:50:25 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/commons-codec_commons-codec-1.11.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp6907334548412285018.tmp
20/08/06 13:50:25 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/commons-codec_commons-codec-1.11.jar to class loader
20/08/06 13:50:25 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.google.protobuf_protobuf-java-util-3.0.0-beta-3.jar with timestamp 1596721808148
20/08/06 13:50:25 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.google.protobuf_protobuf-java-util-3.0.0-beta-3.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp5156821187343029758.tmp
20/08/06 13:50:25 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.google.protobuf_protobuf-java-util-3.0.0-beta-3.jar to class loader
20/08/06 13:50:25 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.spark-project.hive_hive-exec-1.2.1.spark2.jar with timestamp 1596721808128
20/08/06 13:50:25 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.spark-project.hive_hive-exec-1.2.1.spark2.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp5662979525109263631.tmp
20/08/06 13:50:25 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.spark-project.hive_hive-exec-1.2.1.spark2.jar to class loader
20/08/06 13:50:25 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/commons-cli_commons-cli-1.2.jar with timestamp 1596721808134
20/08/06 13:50:25 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/commons-cli_commons-cli-1.2.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp5201877776818030293.tmp
20/08/06 13:50:25 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/commons-cli_commons-cli-1.2.jar to class loader
20/08/06 13:50:25 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.google.code.findbugs_jsr305-3.0.1.jar with timestamp 1596721808150
20/08/06 13:50:25 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.google.code.findbugs_jsr305-3.0.1.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp8318764126141480504.tmp
20/08/06 13:50:25 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.google.code.findbugs_jsr305-3.0.1.jar to class loader
20/08/06 13:50:25 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.amazonaws_aws-java-sdk-kms-1.11.603.jar with timestamp 1596721808146
20/08/06 13:50:25 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.amazonaws_aws-java-sdk-kms-1.11.603.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp4017558083737367064.tmp
20/08/06 13:50:25 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.amazonaws_aws-java-sdk-kms-1.11.603.jar to class loader
20/08/06 13:50:25 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/spark-nlp-starter-fix.jar with timestamp 1596721808151
20/08/06 13:50:25 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/spark-nlp-starter-fix.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp7251554950946967607.tmp
20/08/06 13:50:25 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/spark-nlp-starter-fix.jar to class loader
20/08/06 13:50:25 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/net.hydromatic_eigenbase-properties-1.1.5.jar with timestamp 1596721808142
20/08/06 13:50:25 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/net.hydromatic_eigenbase-properties-1.1.5.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp2144228062226378318.tmp
20/08/06 13:50:25 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/net.hydromatic_eigenbase-properties-1.1.5.jar to class loader
20/08/06 13:50:25 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.typesafe_config-1.3.0.jar with timestamp 1596721808142
20/08/06 13:50:25 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.typesafe_config-1.3.0.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp6082354670068440596.tmp
20/08/06 13:50:25 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.typesafe_config-1.3.0.jar to class loader
20/08/06 13:50:25 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.tensorflow_libtensorflow_jni-1.15.0.jar with timestamp 1596721808151
20/08/06 13:50:25 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.tensorflow_libtensorflow_jni-1.15.0.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp3539498237225777661.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.tensorflow_libtensorflow_jni-1.15.0.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.codehaus.jackson_jackson-mapper-asl-1.9.13.jar with timestamp 1596721808132
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.codehaus.jackson_jackson-mapper-asl-1.9.13.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp6643724708374266580.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.codehaus.jackson_jackson-mapper-asl-1.9.13.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/commons-lang_commons-lang-2.6.jar with timestamp 1596721808133
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/commons-lang_commons-lang-2.6.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp6299858608289990909.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/commons-lang_commons-lang-2.6.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.apache.calcite_calcite-core-1.2.0-incubating.jar with timestamp 1596721808132
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.apache.calcite_calcite-core-1.2.0-incubating.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp2192348890686657692.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.apache.calcite_calcite-core-1.2.0-incubating.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/net.sf.trove4j_trove4j-3.0.3.jar with timestamp 1596721808144
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/net.sf.trove4j_trove4j-3.0.3.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp4025631296047172623.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/net.sf.trove4j_trove4j-3.0.3.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.thoughtworks.paranamer_paranamer-2.8.jar with timestamp 1596721808134
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.thoughtworks.paranamer_paranamer-2.8.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp2537979525656179615.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.thoughtworks.paranamer_paranamer-2.8.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.github.universal-automata_liblevenshtein-3.0.0.jar with timestamp 1596721808143
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.github.universal-automata_liblevenshtein-3.0.0.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp8454956315605193764.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.github.universal-automata_liblevenshtein-3.0.0.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.apache.httpcomponents_httpcore-4.4.11.jar with timestamp 1596721808146
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.apache.httpcomponents_httpcore-4.4.11.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp6557589377655142454.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.apache.httpcomponents_httpcore-4.4.11.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.json4s_json4s-ext_2.11-3.5.3.jar with timestamp 1596721808144
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.json4s_json4s-ext_2.11-3.5.3.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp4777995614597546262.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.json4s_json4s-ext_2.11-3.5.3.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.joda_joda-convert-1.8.1.jar with timestamp 1596721808151
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.joda_joda-convert-1.8.1.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp7932780666830867175.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.joda_joda-convert-1.8.1.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.spark-project.spark_unused-1.0.0.jar with timestamp 1596721808133
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.spark-project.spark_unused-1.0.0.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp4192510468336584649.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.spark-project.spark_unused-1.0.0.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.apache.calcite_calcite-linq4j-1.2.0-incubating.jar with timestamp 1596721808142
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.apache.calcite_calcite-linq4j-1.2.0-incubating.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp61812815655331410.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.apache.calcite_calcite-linq4j-1.2.0-incubating.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.fasterxml.jackson.dataformat_jackson-dataformat-cbor-2.6.7.jar with timestamp 1596721808145
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.fasterxml.jackson.dataformat_jackson-dataformat-cbor-2.6.7.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp4848735520391533181.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.fasterxml.jackson.dataformat_jackson-dataformat-cbor-2.6.7.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.apache.avro_avro-ipc-1.8.2.jar with timestamp 1596721808141
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.apache.avro_avro-ipc-1.8.2.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp6016443175275866516.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.apache.avro_avro-ipc-1.8.2.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/net.jcip_jcip-annotations-1.0.jar with timestamp 1596721808150
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/net.jcip_jcip-annotations-1.0.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp6495931866084016441.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/net.jcip_jcip-annotations-1.0.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.apache.derby_derby-10.12.1.1.jar with timestamp 1596721808132
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.apache.derby_derby-10.12.1.1.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp1603728048453913687.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.apache.derby_derby-10.12.1.1.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.spark-project.hive_hive-metastore-1.2.1.spark2.jar with timestamp 1596721808128
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.spark-project.hive_hive-metastore-1.2.1.spark2.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp6710373346203333298.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.spark-project.hive_hive-metastore-1.2.1.spark2.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.apache.ivy_ivy-2.4.0.jar with timestamp 1596721808133
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.apache.ivy_ivy-2.4.0.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp5208991981056746889.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.apache.ivy_ivy-2.4.0.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.apache.avro_avro-1.8.2.jar with timestamp 1596721808128
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.apache.avro_avro-1.8.2.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp3997920784076660511.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.apache.avro_avro-1.8.2.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.datanucleus_datanucleus-api-jdo-3.2.6.jar with timestamp 1596721808134
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.datanucleus_datanucleus-api-jdo-3.2.6.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp5482656760683992585.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.datanucleus_datanucleus-api-jdo-3.2.6.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.apache.thrift_libfb303-0.9.3.jar with timestamp 1596721808132
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.apache.thrift_libfb303-0.9.3.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp7755514362351325347.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.apache.thrift_libfb303-0.9.3.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.fasterxml.jackson.core_jackson-core-2.6.7.jar with timestamp 1596721808147
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.fasterxml.jackson.core_jackson-core-2.6.7.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp7031663850351946511.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.fasterxml.jackson.core_jackson-core-2.6.7.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/antlr_antlr-2.7.7.jar with timestamp 1596721808134
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/antlr_antlr-2.7.7.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp1997633793580092673.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/antlr_antlr-2.7.7.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/commons-io_commons-io-2.4.jar with timestamp 1596721808133
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/commons-io_commons-io-2.4.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp2265635835106533066.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/commons-io_commons-io-2.4.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.antlr_stringtemplate-3.2.1.jar with timestamp 1596721808134
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.antlr_stringtemplate-3.2.1.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp6832866258324443693.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.antlr_stringtemplate-3.2.1.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/dk.brics.automaton_automaton-1.11-8.jar with timestamp 1596721808151
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/dk.brics.automaton_automaton-1.11-8.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp7173049372206475329.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/dk.brics.automaton_automaton-1.11-8.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.twitter_parquet-hadoop-bundle-1.6.0.jar with timestamp 1596721808128
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.twitter_parquet-hadoop-bundle-1.6.0.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp5523837125970101258.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.twitter_parquet-hadoop-bundle-1.6.0.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.amazonaws_jmespath-java-1.11.603.jar with timestamp 1596721808147
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.amazonaws_jmespath-java-1.11.603.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp4588058963307079748.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.amazonaws_jmespath-java-1.11.603.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.tukaani_xz-1.5.jar with timestamp 1596721808134
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.tukaani_xz-1.5.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp4036502549640396386.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.tukaani_xz-1.5.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.apache.hadoop_hadoop-aws-3.2.0.jar with timestamp 1596721808143
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.apache.hadoop_hadoop-aws-3.2.0.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp7238095967648569440.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.apache.hadoop_hadoop-aws-3.2.0.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.datanucleus_datanucleus-rdbms-3.2.9.jar with timestamp 1596721808139
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.datanucleus_datanucleus-rdbms-3.2.9.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp3445126767812125154.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.datanucleus_datanucleus-rdbms-3.2.9.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/log4j_apache-log4j-extras-1.2.17.jar with timestamp 1596721808133
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/log4j_apache-log4j-extras-1.2.17.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp4949668274565236135.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/log4j_apache-log4j-extras-1.2.17.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/commons-pool_commons-pool-1.5.4.jar with timestamp 1596721808139
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/commons-pool_commons-pool-1.5.4.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp6245573874017033467.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/commons-pool_commons-pool-1.5.4.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/javax.transaction_jta-1.1.jar with timestamp 1596721808141
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/javax.transaction_jta-1.1.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp7923997938778769974.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/javax.transaction_jta-1.1.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.slf4j_slf4j-api-1.7.21.jar with timestamp 1596721808150
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.slf4j_slf4j-api-1.7.21.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp1330948957194041169.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.slf4j_slf4j-api-1.7.21.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/joda-time_joda-time-2.9.5.jar with timestamp 1596721808151
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/joda-time_joda-time-2.9.5.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp789829430378624239.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/joda-time_joda-time-2.9.5.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/javax.jdo_jdo-api-3.0.1.jar with timestamp 1596721808140
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/javax.jdo_jdo-api-3.0.1.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp4515772672548043226.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/javax.jdo_jdo-api-3.0.1.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.codehaus.jackson_jackson-core-asl-1.9.13.jar with timestamp 1596721808133
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.codehaus.jackson_jackson-core-asl-1.9.13.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp8070247797080486124.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.codehaus.jackson_jackson-core-asl-1.9.13.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.jodd_jodd-core-3.5.2.jar with timestamp 1596721808132
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.jodd_jodd-core-3.5.2.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp2455687742329803272.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.jodd_jodd-core-3.5.2.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.fasterxml.jackson.core_jackson-databind-2.6.7.2.jar with timestamp 1596721808147
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.fasterxml.jackson.core_jackson-databind-2.6.7.2.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp704537551408706052.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.fasterxml.jackson.core_jackson-databind-2.6.7.2.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.apache.calcite_calcite-avatica-1.2.0-incubating.jar with timestamp 1596721808132
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.apache.calcite_calcite-avatica-1.2.0-incubating.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp3773777591182670488.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.apache.calcite_calcite-avatica-1.2.0-incubating.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.projectlombok_lombok-1.16.8.jar with timestamp 1596721808149
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.projectlombok_lombok-1.16.8.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp7732153328532786227.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.projectlombok_lombok-1.16.8.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.rocksdb_rocksdbjni-6.5.3.jar with timestamp 1596721808143
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.rocksdb_rocksdbjni-6.5.3.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp4059064783124798536.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.rocksdb_rocksdbjni-6.5.3.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/commons-dbcp_commons-dbcp-1.4.jar with timestamp 1596721808140
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/commons-dbcp_commons-dbcp-1.4.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp2653719113988241420.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/commons-dbcp_commons-dbcp-1.4.jar to class loader
20/08/06 13:50:26 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/it.unimi.dsi_fastutil-7.0.12.jar with timestamp 1596721808149
20/08/06 13:50:26 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/it.unimi.dsi_fastutil-7.0.12.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp6008606279688713538.tmp
20/08/06 13:50:26 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/it.unimi.dsi_fastutil-7.0.12.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.johnsnowlabs.nlp_spark-nlp_2.11-2.5.4.jar with timestamp 1596721808127
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.johnsnowlabs.nlp_spark-nlp_2.11-2.5.4.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp4812817873370372460.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.johnsnowlabs.nlp_spark-nlp_2.11-2.5.4.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.tensorflow_libtensorflow-1.15.0.jar with timestamp 1596721808151
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.tensorflow_libtensorflow-1.15.0.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp5247162954889159346.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.tensorflow_libtensorflow-1.15.0.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.google.code.gson_gson-2.3.jar with timestamp 1596721808151
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.google.code.gson_gson-2.3.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp3701346576751178937.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.google.code.gson_gson-2.3.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.xerial.snappy_snappy-java-1.1.7.1.jar with timestamp 1596721808134
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.xerial.snappy_snappy-java-1.1.7.1.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp7902927617406233479.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.xerial.snappy_snappy-java-1.1.7.1.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.amazonaws_aws-java-sdk-core-1.11.603.jar with timestamp 1596721808143
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.amazonaws_aws-java-sdk-core-1.11.603.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp5389390534250652569.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.amazonaws_aws-java-sdk-core-1.11.603.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.fasterxml.jackson.core_jackson-annotations-2.6.0.jar with timestamp 1596721808147
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.fasterxml.jackson.core_jackson-annotations-2.6.0.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp4247467263900316127.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.fasterxml.jackson.core_jackson-annotations-2.6.0.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.tensorflow_tensorflow-1.15.0.jar with timestamp 1596721808144
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.tensorflow_tensorflow-1.15.0.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp4398141401768571179.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.tensorflow_tensorflow-1.15.0.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.apache.commons_commons-lang3-3.5.jar with timestamp 1596721808133
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.apache.commons_commons-lang3-3.5.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp8121122923310758350.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.apache.commons_commons-lang3-3.5.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.google.code.findbugs_annotations-3.0.1.jar with timestamp 1596721808147
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.google.code.findbugs_annotations-3.0.1.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp9109790451347104855.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.google.code.findbugs_annotations-3.0.1.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/stax_stax-api-1.0.1.jar with timestamp 1596721808133
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/stax_stax-api-1.0.1.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp6638842456947076493.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/stax_stax-api-1.0.1.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.antlr_antlr-runtime-3.4.jar with timestamp 1596721808133
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.antlr_antlr-runtime-3.4.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp7489236121094269006.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.antlr_antlr-runtime-3.4.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.google.protobuf_protobuf-java-3.0.0-beta-3.jar with timestamp 1596721808148
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.google.protobuf_protobuf-java-3.0.0-beta-3.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp6089113642068714764.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.google.protobuf_protobuf-java-3.0.0-beta-3.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.navigamez_greex-1.0.jar with timestamp 1596721808144
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.navigamez_greex-1.0.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp4563679365120543575.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.navigamez_greex-1.0.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.antlr_ST4-4.0.4.jar with timestamp 1596721808133
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.antlr_ST4-4.0.4.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp5657485864061339928.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.antlr_ST4-4.0.4.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.apache.spark_spark-hive_2.11-2.4.0.jar with timestamp 1596721808126
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.apache.spark_spark-hive_2.11-2.4.0.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp3620220856941053338.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.apache.spark_spark-hive_2.11-2.4.0.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.datanucleus_datanucleus-core-3.2.10.jar with timestamp 1596721808132
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.datanucleus_datanucleus-core-3.2.10.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp6588388166475833286.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.datanucleus_datanucleus-core-3.2.10.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/com.jolbox_bonecp-0.8.0.RELEASE.jar with timestamp 1596721808134
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/com.jolbox_bonecp-0.8.0.RELEASE.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp7914950543583503207.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/com.jolbox_bonecp-0.8.0.RELEASE.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.iq80.snappy_snappy-0.2.jar with timestamp 1596721808133
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.iq80.snappy_snappy-0.2.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp4912967906438420984.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.iq80.snappy_snappy-0.2.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/commons-logging_commons-logging-1.2.jar with timestamp 1596721808142
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/commons-logging_commons-logging-1.2.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp9205044888967033296.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/commons-logging_commons-logging-1.2.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.apache.thrift_libthrift-0.9.3.jar with timestamp 1596721808132
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.apache.thrift_libthrift-0.9.3.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp3020080351613423848.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.apache.thrift_libthrift-0.9.3.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.apache.httpcomponents_httpclient-4.5.9.jar with timestamp 1596721808145
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.apache.httpcomponents_httpclient-4.5.9.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp7420726860167371449.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.apache.httpcomponents_httpclient-4.5.9.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/commons-httpclient_commons-httpclient-3.1.jar with timestamp 1596721808132
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/commons-httpclient_commons-httpclient-3.1.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp5549169553072435891.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/commons-httpclient_commons-httpclient-3.1.jar to class loader
20/08/06 13:50:27 INFO Executor: Fetching spark://56f1369e0bb4:38823/jars/org.apache.commons_commons-compress-1.4.1.jar with timestamp 1596721808133
20/08/06 13:50:27 INFO Utils: Fetching spark://56f1369e0bb4:38823/jars/org.apache.commons_commons-compress-1.4.1.jar to /tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/fetchFileTemp9064267229070157597.tmp
20/08/06 13:50:27 INFO Executor: Adding file:/tmp/spark-d68e60c9-9470-4b52-b669-7d4982894782/userFiles-fba9215a-0c6c-45a6-b7a5-40e860297e6f/org.apache.commons_commons-compress-1.4.1.jar to class loader
20/08/06 13:50:27 INFO HadoopRDD: Input split: file:/root/cache_pretrained/glove_100d_en_2.4.0_2.4_1579690104032/metadata/part-00000:0+377
20/08/06 13:50:27 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 1219 bytes result sent to driver
20/08/06 13:50:27 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 1919 ms on localhost (executor driver) (1/1)
20/08/06 13:50:27 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool 
20/08/06 13:50:27 INFO DAGScheduler: ResultStage 0 (first at ReadWrite.scala:615) finished in 2.014 s
20/08/06 13:50:27 INFO DAGScheduler: Job 0 finished: first at ReadWrite.scala:615, took 2.093151 s
ner_dl download started this may take some time.
20/08/06 13:50:27 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
20/08/06 13:50:27 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
20/08/06 13:50:27 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
20/08/06 13:50:27 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
20/08/06 13:50:28 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
20/08/06 13:50:28 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
Approximate size to download 13.6 MB
20/08/06 13:50:28 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
20/08/06 13:50:28 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
20/08/06 13:50:28 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
20/08/06 13:50:28 WARN ApacheUtils: NoSuchMethodError was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See https://github.com/aws/aws-sdk-java/issues/1919 for more information
Download done! Loading the resource.
20/08/06 13:50:31 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 236.7 KB, free 2004.1 MB)
20/08/06 13:50:31 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 22.9 KB, free 2004.1 MB)
20/08/06 13:50:31 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on 56f1369e0bb4:43539 (size: 22.9 KB, free: 2004.6 MB)
20/08/06 13:50:31 INFO SparkContext: Created broadcast 2 from textFile at ReadWrite.scala:615
20/08/06 13:50:31 INFO FileInputFormat: Total input paths to process : 1
20/08/06 13:50:31 INFO SparkContext: Starting job: first at ReadWrite.scala:615
20/08/06 13:50:31 INFO DAGScheduler: Got job 1 (first at ReadWrite.scala:615) with 1 output partitions
20/08/06 13:50:31 INFO DAGScheduler: Final stage: ResultStage 1 (first at ReadWrite.scala:615)
20/08/06 13:50:31 INFO DAGScheduler: Parents of final stage: List()
20/08/06 13:50:31 INFO DAGScheduler: Missing parents: List()
20/08/06 13:50:31 INFO DAGScheduler: Submitting ResultStage 1 (file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/metadata MapPartitionsRDD[3] at textFile at ReadWrite.scala:615), which has no missing parents
20/08/06 13:50:31 INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 3.5 KB, free 2004.1 MB)
20/08/06 13:50:31 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 2.1 KB, free 2004.1 MB)
20/08/06 13:50:31 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on 56f1369e0bb4:43539 (size: 2.1 KB, free: 2004.6 MB)
20/08/06 13:50:31 INFO SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:1161
20/08/06 13:50:31 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 1 (file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/metadata MapPartitionsRDD[3] at textFile at ReadWrite.scala:615) (first 15 tasks are for partitions Vector(0))
20/08/06 13:50:31 INFO TaskSchedulerImpl: Adding task set 1.0 with 1 tasks
20/08/06 13:50:31 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 1, localhost, executor driver, partition 0, PROCESS_LOCAL, 7941 bytes)
20/08/06 13:50:31 INFO Executor: Running task 0.0 in stage 1.0 (TID 1)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/metadata/part-00000:0+408
20/08/06 13:50:31 INFO Executor: Finished task 0.0 in stage 1.0 (TID 1). 1207 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 1) in 16 ms on localhost (executor driver) (1/1)
20/08/06 13:50:31 INFO DAGScheduler: ResultStage 1 (first at ReadWrite.scala:615) finished in 0.023 s
20/08/06 13:50:31 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool 
20/08/06 13:50:31 INFO DAGScheduler: Job 1 finished: first at ReadWrite.scala:615, took 0.026184 s
20/08/06 13:50:31 INFO MemoryStore: Block broadcast_4 stored as values in memory (estimated size 236.7 KB, free 2003.9 MB)
20/08/06 13:50:31 INFO MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 22.9 KB, free 2003.8 MB)
20/08/06 13:50:31 INFO BlockManagerInfo: Added broadcast_4_piece0 in memory on 56f1369e0bb4:43539 (size: 22.9 KB, free: 2004.5 MB)
20/08/06 13:50:31 INFO SparkContext: Created broadcast 4 from objectFile at Feature.scala:119
20/08/06 13:50:31 INFO FileInputFormat: Total input paths to process : 40
20/08/06 13:50:31 INFO SparkContext: Starting job: first at Feature.scala:119
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 30
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 25
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 49
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 40
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 43
20/08/06 13:50:31 INFO DAGScheduler: Got job 2 (first at Feature.scala:119) with 1 output partitions
20/08/06 13:50:31 INFO DAGScheduler: Final stage: ResultStage 2 (first at Feature.scala:119)
20/08/06 13:50:31 INFO DAGScheduler: Parents of final stage: List()
20/08/06 13:50:31 INFO DAGScheduler: Missing parents: List()
20/08/06 13:50:31 INFO DAGScheduler: Submitting ResultStage 2 (MapPartitionsRDD[5] at objectFile at Feature.scala:119), which has no missing parents
20/08/06 13:50:31 INFO MemoryStore: Block broadcast_5 stored as values in memory (estimated size 3.7 KB, free 2003.8 MB)
20/08/06 13:50:31 INFO MemoryStore: Block broadcast_5_piece0 stored as bytes in memory (estimated size 2.2 KB, free 2003.8 MB)
20/08/06 13:50:31 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on 56f1369e0bb4:43539 (size: 2.2 KB, free: 2004.5 MB)
20/08/06 13:50:31 INFO SparkContext: Created broadcast 5 from broadcast at DAGScheduler.scala:1161
20/08/06 13:50:31 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 2 (MapPartitionsRDD[5] at objectFile at Feature.scala:119) (first 15 tasks are for partitions Vector(0))
20/08/06 13:50:31 INFO TaskSchedulerImpl: Adding task set 2.0 with 1 tasks
20/08/06 13:50:31 INFO TaskSetManager: Starting task 0.0 in stage 2.0 (TID 2, localhost, executor driver, partition 0, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO Executor: Running task 0.0 in stage 2.0 (TID 2)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00011:0+95
20/08/06 13:50:31 INFO BlockManagerInfo: Removed broadcast_3_piece0 on 56f1369e0bb4:43539 in memory (size: 2.1 KB, free: 2004.5 MB)
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 34
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 47
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 29
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 28
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 39
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 45
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 27
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 32
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 38
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 33
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 26
20/08/06 13:50:31 INFO BlockManagerInfo: Removed broadcast_2_piece0 on 56f1369e0bb4:43539 in memory (size: 22.9 KB, free: 2004.6 MB)
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 31
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 36
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 48
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 37
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 44
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 35
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 46
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 41
20/08/06 13:50:31 INFO ContextCleaner: Cleaned accumulator 42
20/08/06 13:50:31 INFO Executor: Finished task 0.0 in stage 2.0 (TID 2). 786 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Finished task 0.0 in stage 2.0 (TID 2) in 37 ms on localhost (executor driver) (1/1)
20/08/06 13:50:31 INFO DAGScheduler: ResultStage 2 (first at Feature.scala:119) finished in 0.047 s
20/08/06 13:50:31 INFO DAGScheduler: Job 2 finished: first at Feature.scala:119, took 0.049810 s
20/08/06 13:50:31 INFO TaskSchedulerImpl: Removed TaskSet 2.0, whose tasks have all completed, from pool 
20/08/06 13:50:31 INFO SparkContext: Starting job: first at Feature.scala:119
20/08/06 13:50:31 INFO DAGScheduler: Got job 3 (first at Feature.scala:119) with 4 output partitions
20/08/06 13:50:31 INFO DAGScheduler: Final stage: ResultStage 3 (first at Feature.scala:119)
20/08/06 13:50:31 INFO DAGScheduler: Parents of final stage: List()
20/08/06 13:50:31 INFO DAGScheduler: Missing parents: List()
20/08/06 13:50:31 INFO DAGScheduler: Submitting ResultStage 3 (MapPartitionsRDD[5] at objectFile at Feature.scala:119), which has no missing parents
20/08/06 13:50:31 INFO MemoryStore: Block broadcast_6 stored as values in memory (estimated size 3.7 KB, free 2004.1 MB)
20/08/06 13:50:31 INFO MemoryStore: Block broadcast_6_piece0 stored as bytes in memory (estimated size 2.2 KB, free 2004.1 MB)
20/08/06 13:50:31 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on 56f1369e0bb4:43539 (size: 2.2 KB, free: 2004.5 MB)
20/08/06 13:50:31 INFO SparkContext: Created broadcast 6 from broadcast at DAGScheduler.scala:1161
20/08/06 13:50:31 INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 3 (MapPartitionsRDD[5] at objectFile at Feature.scala:119) (first 15 tasks are for partitions Vector(1, 2, 3, 4))
20/08/06 13:50:31 INFO TaskSchedulerImpl: Adding task set 3.0 with 4 tasks
20/08/06 13:50:31 INFO TaskSetManager: Starting task 0.0 in stage 3.0 (TID 3, localhost, executor driver, partition 1, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 1.0 in stage 3.0 (TID 4, localhost, executor driver, partition 2, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 2.0 in stage 3.0 (TID 5, localhost, executor driver, partition 3, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 3.0 in stage 3.0 (TID 6, localhost, executor driver, partition 4, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO Executor: Running task 0.0 in stage 3.0 (TID 3)
20/08/06 13:50:31 INFO Executor: Running task 1.0 in stage 3.0 (TID 4)
20/08/06 13:50:31 INFO Executor: Running task 2.0 in stage 3.0 (TID 5)
20/08/06 13:50:31 INFO Executor: Running task 3.0 in stage 3.0 (TID 6)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00001:0+95
20/08/06 13:50:31 INFO Executor: Finished task 0.0 in stage 3.0 (TID 3). 786 bytes result sent to driver
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00036:0+95
20/08/06 13:50:31 INFO TaskSetManager: Finished task 0.0 in stage 3.0 (TID 3) in 20 ms on localhost (executor driver) (1/4)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00024:0+95
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00009:0+95
20/08/06 13:50:31 INFO Executor: Finished task 3.0 in stage 3.0 (TID 6). 786 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Finished task 3.0 in stage 3.0 (TID 6) in 36 ms on localhost (executor driver) (2/4)
20/08/06 13:50:31 INFO Executor: Finished task 2.0 in stage 3.0 (TID 5). 786 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Finished task 2.0 in stage 3.0 (TID 5) in 38 ms on localhost (executor driver) (3/4)
20/08/06 13:50:31 INFO Executor: Finished task 1.0 in stage 3.0 (TID 4). 786 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Finished task 1.0 in stage 3.0 (TID 4) in 40 ms on localhost (executor driver) (4/4)
20/08/06 13:50:31 INFO TaskSchedulerImpl: Removed TaskSet 3.0, whose tasks have all completed, from pool 
20/08/06 13:50:31 INFO DAGScheduler: ResultStage 3 (first at Feature.scala:119) finished in 0.049 s
20/08/06 13:50:31 INFO DAGScheduler: Job 3 finished: first at Feature.scala:119, took 0.052412 s
20/08/06 13:50:31 INFO SparkContext: Starting job: first at Feature.scala:119
20/08/06 13:50:31 INFO DAGScheduler: Got job 4 (first at Feature.scala:119) with 20 output partitions
20/08/06 13:50:31 INFO DAGScheduler: Final stage: ResultStage 4 (first at Feature.scala:119)
20/08/06 13:50:31 INFO DAGScheduler: Parents of final stage: List()
20/08/06 13:50:31 INFO DAGScheduler: Missing parents: List()
20/08/06 13:50:31 INFO DAGScheduler: Submitting ResultStage 4 (MapPartitionsRDD[5] at objectFile at Feature.scala:119), which has no missing parents
20/08/06 13:50:31 INFO MemoryStore: Block broadcast_7 stored as values in memory (estimated size 3.7 KB, free 2004.1 MB)
20/08/06 13:50:31 INFO MemoryStore: Block broadcast_7_piece0 stored as bytes in memory (estimated size 2.2 KB, free 2004.1 MB)
20/08/06 13:50:31 INFO BlockManagerInfo: Added broadcast_7_piece0 in memory on 56f1369e0bb4:43539 (size: 2.2 KB, free: 2004.5 MB)
20/08/06 13:50:31 INFO SparkContext: Created broadcast 7 from broadcast at DAGScheduler.scala:1161
20/08/06 13:50:31 INFO DAGScheduler: Submitting 20 missing tasks from ResultStage 4 (MapPartitionsRDD[5] at objectFile at Feature.scala:119) (first 15 tasks are for partitions Vector(5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19))
20/08/06 13:50:31 INFO TaskSchedulerImpl: Adding task set 4.0 with 20 tasks
20/08/06 13:50:31 INFO TaskSetManager: Starting task 0.0 in stage 4.0 (TID 7, localhost, executor driver, partition 5, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 1.0 in stage 4.0 (TID 8, localhost, executor driver, partition 6, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 2.0 in stage 4.0 (TID 9, localhost, executor driver, partition 7, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 3.0 in stage 4.0 (TID 10, localhost, executor driver, partition 8, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO Executor: Running task 0.0 in stage 4.0 (TID 7)
20/08/06 13:50:31 INFO Executor: Running task 1.0 in stage 4.0 (TID 8)
20/08/06 13:50:31 INFO Executor: Running task 2.0 in stage 4.0 (TID 9)
20/08/06 13:50:31 INFO Executor: Running task 3.0 in stage 4.0 (TID 10)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00014:0+95
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00003:0+95
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00033:0+95
20/08/06 13:50:31 INFO Executor: Finished task 3.0 in stage 4.0 (TID 10). 786 bytes result sent to driver
20/08/06 13:50:31 INFO Executor: Finished task 0.0 in stage 4.0 (TID 7). 786 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Starting task 4.0 in stage 4.0 (TID 11, localhost, executor driver, partition 9, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 3.0 in stage 4.0 (TID 10) in 10 ms on localhost (executor driver) (1/20)
20/08/06 13:50:31 INFO Executor: Finished task 2.0 in stage 4.0 (TID 9). 786 bytes result sent to driver
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00034:0+95
20/08/06 13:50:31 INFO TaskSetManager: Starting task 5.0 in stage 4.0 (TID 12, localhost, executor driver, partition 10, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO Executor: Running task 5.0 in stage 4.0 (TID 12)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 6.0 in stage 4.0 (TID 13, localhost, executor driver, partition 11, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 2.0 in stage 4.0 (TID 9) in 18 ms on localhost (executor driver) (2/20)
20/08/06 13:50:31 INFO Executor: Finished task 1.0 in stage 4.0 (TID 8). 786 bytes result sent to driver
20/08/06 13:50:31 INFO Executor: Running task 4.0 in stage 4.0 (TID 11)
20/08/06 13:50:31 INFO Executor: Running task 6.0 in stage 4.0 (TID 13)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 0.0 in stage 4.0 (TID 7) in 19 ms on localhost (executor driver) (3/20)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 7.0 in stage 4.0 (TID 14, localhost, executor driver, partition 12, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 1.0 in stage 4.0 (TID 8) in 19 ms on localhost (executor driver) (4/20)
20/08/06 13:50:31 INFO Executor: Running task 7.0 in stage 4.0 (TID 14)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00015:0+95
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00013:0+95
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00004:0+95
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00017:0+95
20/08/06 13:50:31 INFO Executor: Finished task 5.0 in stage 4.0 (TID 12). 786 bytes result sent to driver
20/08/06 13:50:31 INFO Executor: Finished task 7.0 in stage 4.0 (TID 14). 786 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Starting task 8.0 in stage 4.0 (TID 15, localhost, executor driver, partition 13, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 7.0 in stage 4.0 (TID 14) in 10 ms on localhost (executor driver) (5/20)
20/08/06 13:50:31 INFO Executor: Running task 8.0 in stage 4.0 (TID 15)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 9.0 in stage 4.0 (TID 16, localhost, executor driver, partition 14, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO Executor: Running task 9.0 in stage 4.0 (TID 16)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 5.0 in stage 4.0 (TID 12) in 16 ms on localhost (executor driver) (6/20)
20/08/06 13:50:31 INFO Executor: Finished task 4.0 in stage 4.0 (TID 11). 786 bytes result sent to driver
20/08/06 13:50:31 INFO Executor: Finished task 6.0 in stage 4.0 (TID 13). 829 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Starting task 10.0 in stage 4.0 (TID 17, localhost, executor driver, partition 15, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 4.0 in stage 4.0 (TID 11) in 26 ms on localhost (executor driver) (7/20)
20/08/06 13:50:31 INFO Executor: Running task 10.0 in stage 4.0 (TID 17)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 11.0 in stage 4.0 (TID 18, localhost, executor driver, partition 16, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 6.0 in stage 4.0 (TID 13) in 21 ms on localhost (executor driver) (8/20)
20/08/06 13:50:31 INFO Executor: Running task 11.0 in stage 4.0 (TID 18)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00006:0+95
20/08/06 13:50:31 INFO Executor: Finished task 8.0 in stage 4.0 (TID 15). 786 bytes result sent to driver
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00032:0+95
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00002:0+95
20/08/06 13:50:31 INFO TaskSetManager: Starting task 12.0 in stage 4.0 (TID 19, localhost, executor driver, partition 17, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 8.0 in stage 4.0 (TID 15) in 19 ms on localhost (executor driver) (9/20)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00008:0+95
20/08/06 13:50:31 INFO Executor: Running task 12.0 in stage 4.0 (TID 19)
20/08/06 13:50:31 INFO Executor: Finished task 11.0 in stage 4.0 (TID 18). 829 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Finished task 11.0 in stage 4.0 (TID 18) in 12 ms on localhost (executor driver) (10/20)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 13.0 in stage 4.0 (TID 20, localhost, executor driver, partition 18, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO Executor: Running task 13.0 in stage 4.0 (TID 20)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00019:0+95
20/08/06 13:50:31 INFO Executor: Finished task 10.0 in stage 4.0 (TID 17). 786 bytes result sent to driver
20/08/06 13:50:31 INFO Executor: Finished task 9.0 in stage 4.0 (TID 16). 786 bytes result sent to driver
20/08/06 13:50:31 INFO Executor: Finished task 12.0 in stage 4.0 (TID 19). 786 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Starting task 14.0 in stage 4.0 (TID 21, localhost, executor driver, partition 19, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 15.0 in stage 4.0 (TID 22, localhost, executor driver, partition 20, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO Executor: Running task 15.0 in stage 4.0 (TID 22)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 16.0 in stage 4.0 (TID 23, localhost, executor driver, partition 21, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 10.0 in stage 4.0 (TID 17) in 23 ms on localhost (executor driver) (11/20)
20/08/06 13:50:31 INFO Executor: Running task 14.0 in stage 4.0 (TID 21)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 12.0 in stage 4.0 (TID 19) in 15 ms on localhost (executor driver) (12/20)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 9.0 in stage 4.0 (TID 16) in 29 ms on localhost (executor driver) (13/20)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00026:0+95
20/08/06 13:50:31 INFO Executor: Running task 16.0 in stage 4.0 (TID 23)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00038:0+95
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00023:0+95
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00005:0+95
20/08/06 13:50:31 INFO Executor: Finished task 15.0 in stage 4.0 (TID 22). 786 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Starting task 17.0 in stage 4.0 (TID 24, localhost, executor driver, partition 22, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 15.0 in stage 4.0 (TID 22) in 11 ms on localhost (executor driver) (14/20)
20/08/06 13:50:31 INFO Executor: Running task 17.0 in stage 4.0 (TID 24)
20/08/06 13:50:31 INFO Executor: Finished task 14.0 in stage 4.0 (TID 21). 829 bytes result sent to driver
20/08/06 13:50:31 INFO Executor: Finished task 16.0 in stage 4.0 (TID 23). 786 bytes result sent to driver
20/08/06 13:50:31 INFO Executor: Finished task 13.0 in stage 4.0 (TID 20). 786 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Starting task 18.0 in stage 4.0 (TID 25, localhost, executor driver, partition 23, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 14.0 in stage 4.0 (TID 21) in 18 ms on localhost (executor driver) (15/20)
20/08/06 13:50:31 INFO Executor: Running task 18.0 in stage 4.0 (TID 25)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 13.0 in stage 4.0 (TID 20) in 23 ms on localhost (executor driver) (16/20)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 19.0 in stage 4.0 (TID 26, localhost, executor driver, partition 24, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO Executor: Running task 19.0 in stage 4.0 (TID 26)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 16.0 in stage 4.0 (TID 23) in 17 ms on localhost (executor driver) (17/20)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00018:0+95
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00031:0+95
20/08/06 13:50:31 INFO Executor: Finished task 18.0 in stage 4.0 (TID 25). 786 bytes result sent to driver
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00010:0+95
20/08/06 13:50:31 INFO TaskSetManager: Finished task 18.0 in stage 4.0 (TID 25) in 9 ms on localhost (executor driver) (18/20)
20/08/06 13:50:31 INFO Executor: Finished task 17.0 in stage 4.0 (TID 24). 786 bytes result sent to driver
20/08/06 13:50:31 INFO Executor: Finished task 19.0 in stage 4.0 (TID 26). 786 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Finished task 17.0 in stage 4.0 (TID 24) in 18 ms on localhost (executor driver) (19/20)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 19.0 in stage 4.0 (TID 26) in 11 ms on localhost (executor driver) (20/20)
20/08/06 13:50:31 INFO TaskSchedulerImpl: Removed TaskSet 4.0, whose tasks have all completed, from pool 
20/08/06 13:50:31 INFO DAGScheduler: ResultStage 4 (first at Feature.scala:119) finished in 0.093 s
20/08/06 13:50:31 INFO DAGScheduler: Job 4 finished: first at Feature.scala:119, took 0.095956 s
20/08/06 13:50:31 INFO SparkContext: Starting job: first at Feature.scala:119
20/08/06 13:50:31 INFO DAGScheduler: Got job 5 (first at Feature.scala:119) with 15 output partitions
20/08/06 13:50:31 INFO DAGScheduler: Final stage: ResultStage 5 (first at Feature.scala:119)
20/08/06 13:50:31 INFO DAGScheduler: Parents of final stage: List()
20/08/06 13:50:31 INFO DAGScheduler: Missing parents: List()
20/08/06 13:50:31 INFO DAGScheduler: Submitting ResultStage 5 (MapPartitionsRDD[5] at objectFile at Feature.scala:119), which has no missing parents
20/08/06 13:50:31 INFO MemoryStore: Block broadcast_8 stored as values in memory (estimated size 3.7 KB, free 2004.1 MB)
20/08/06 13:50:31 INFO MemoryStore: Block broadcast_8_piece0 stored as bytes in memory (estimated size 2.2 KB, free 2004.1 MB)
20/08/06 13:50:31 INFO BlockManagerInfo: Added broadcast_8_piece0 in memory on 56f1369e0bb4:43539 (size: 2.2 KB, free: 2004.5 MB)
20/08/06 13:50:31 INFO SparkContext: Created broadcast 8 from broadcast at DAGScheduler.scala:1161
20/08/06 13:50:31 INFO DAGScheduler: Submitting 15 missing tasks from ResultStage 5 (MapPartitionsRDD[5] at objectFile at Feature.scala:119) (first 15 tasks are for partitions Vector(25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39))
20/08/06 13:50:31 INFO TaskSchedulerImpl: Adding task set 5.0 with 15 tasks
20/08/06 13:50:31 INFO TaskSetManager: Starting task 0.0 in stage 5.0 (TID 27, localhost, executor driver, partition 25, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 1.0 in stage 5.0 (TID 28, localhost, executor driver, partition 26, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 2.0 in stage 5.0 (TID 29, localhost, executor driver, partition 27, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 3.0 in stage 5.0 (TID 30, localhost, executor driver, partition 28, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO Executor: Running task 0.0 in stage 5.0 (TID 27)
20/08/06 13:50:31 INFO Executor: Running task 3.0 in stage 5.0 (TID 30)
20/08/06 13:50:31 INFO Executor: Running task 2.0 in stage 5.0 (TID 29)
20/08/06 13:50:31 INFO Executor: Running task 1.0 in stage 5.0 (TID 28)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00016:0+95
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00029:0+95
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00030:0+95
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00037:0+95
20/08/06 13:50:31 INFO Executor: Finished task 2.0 in stage 5.0 (TID 29). 786 bytes result sent to driver
20/08/06 13:50:31 INFO Executor: Finished task 1.0 in stage 5.0 (TID 28). 786 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Starting task 4.0 in stage 5.0 (TID 31, localhost, executor driver, partition 29, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 2.0 in stage 5.0 (TID 29) in 20 ms on localhost (executor driver) (1/15)
20/08/06 13:50:31 INFO Executor: Running task 4.0 in stage 5.0 (TID 31)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 5.0 in stage 5.0 (TID 32, localhost, executor driver, partition 30, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 1.0 in stage 5.0 (TID 28) in 22 ms on localhost (executor driver) (2/15)
20/08/06 13:50:31 INFO Executor: Running task 5.0 in stage 5.0 (TID 32)
20/08/06 13:50:31 INFO Executor: Finished task 3.0 in stage 5.0 (TID 30). 786 bytes result sent to driver
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00007:0+95
20/08/06 13:50:31 INFO TaskSetManager: Starting task 6.0 in stage 5.0 (TID 33, localhost, executor driver, partition 31, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00000:0+95
20/08/06 13:50:31 INFO Executor: Running task 6.0 in stage 5.0 (TID 33)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 3.0 in stage 5.0 (TID 30) in 25 ms on localhost (executor driver) (3/15)
20/08/06 13:50:31 INFO Executor: Finished task 0.0 in stage 5.0 (TID 27). 829 bytes result sent to driver
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00025:0+95
20/08/06 13:50:31 INFO TaskSetManager: Starting task 7.0 in stage 5.0 (TID 34, localhost, executor driver, partition 32, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO Executor: Running task 7.0 in stage 5.0 (TID 34)
20/08/06 13:50:31 INFO Executor: Finished task 6.0 in stage 5.0 (TID 33). 786 bytes result sent to driver
20/08/06 13:50:31 INFO Executor: Finished task 5.0 in stage 5.0 (TID 32). 786 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Finished task 0.0 in stage 5.0 (TID 27) in 35 ms on localhost (executor driver) (4/15)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00012:0+95
20/08/06 13:50:31 INFO TaskSetManager: Starting task 8.0 in stage 5.0 (TID 35, localhost, executor driver, partition 33, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO Executor: Running task 8.0 in stage 5.0 (TID 35)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 9.0 in stage 5.0 (TID 36, localhost, executor driver, partition 34, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO Executor: Running task 9.0 in stage 5.0 (TID 36)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 6.0 in stage 5.0 (TID 33) in 14 ms on localhost (executor driver) (5/15)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 5.0 in stage 5.0 (TID 32) in 19 ms on localhost (executor driver) (6/15)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00027:0+95
20/08/06 13:50:31 INFO Executor: Finished task 7.0 in stage 5.0 (TID 34). 786 bytes result sent to driver
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00020:0+95
20/08/06 13:50:31 INFO TaskSetManager: Starting task 10.0 in stage 5.0 (TID 37, localhost, executor driver, partition 35, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO Executor: Finished task 8.0 in stage 5.0 (TID 35). 786 bytes result sent to driver
20/08/06 13:50:31 INFO Executor: Finished task 4.0 in stage 5.0 (TID 31). 786 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Finished task 7.0 in stage 5.0 (TID 34) in 10 ms on localhost (executor driver) (7/15)
20/08/06 13:50:31 INFO Executor: Running task 10.0 in stage 5.0 (TID 37)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 11.0 in stage 5.0 (TID 38, localhost, executor driver, partition 36, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 4.0 in stage 5.0 (TID 31) in 26 ms on localhost (executor driver) (8/15)
20/08/06 13:50:31 INFO Executor: Running task 11.0 in stage 5.0 (TID 38)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 8.0 in stage 5.0 (TID 35) in 8 ms on localhost (executor driver) (9/15)
20/08/06 13:50:31 INFO TaskSetManager: Starting task 12.0 in stage 5.0 (TID 39, localhost, executor driver, partition 37, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO Executor: Running task 12.0 in stage 5.0 (TID 39)
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00022:0+95
20/08/06 13:50:31 INFO Executor: Finished task 9.0 in stage 5.0 (TID 36). 786 bytes result sent to driver
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00021:0+95
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00028:0+95
20/08/06 13:50:31 INFO TaskSetManager: Starting task 13.0 in stage 5.0 (TID 40, localhost, executor driver, partition 38, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO Executor: Running task 13.0 in stage 5.0 (TID 40)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 9.0 in stage 5.0 (TID 36) in 12 ms on localhost (executor driver) (10/15)
20/08/06 13:50:31 INFO Executor: Finished task 10.0 in stage 5.0 (TID 37). 786 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Starting task 14.0 in stage 5.0 (TID 41, localhost, executor driver, partition 39, PROCESS_LOCAL, 7953 bytes)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 10.0 in stage 5.0 (TID 37) in 9 ms on localhost (executor driver) (11/15)
20/08/06 13:50:31 INFO Executor: Running task 14.0 in stage 5.0 (TID 41)
20/08/06 13:50:31 INFO Executor: Finished task 11.0 in stage 5.0 (TID 38). 829 bytes result sent to driver
20/08/06 13:50:31 INFO Executor: Finished task 12.0 in stage 5.0 (TID 39). 786 bytes result sent to driver
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00035:0+95
20/08/06 13:50:31 INFO HadoopRDD: Input split: file:/root/cache_pretrained/ner_dl_en_2.4.3_2.4_1584624950746/fields/datasetParams/part-00039:0+2831
20/08/06 13:50:31 INFO TaskSetManager: Finished task 12.0 in stage 5.0 (TID 39) in 15 ms on localhost (executor driver) (12/15)
20/08/06 13:50:31 INFO TaskSetManager: Finished task 11.0 in stage 5.0 (TID 38) in 23 ms on localhost (executor driver) (13/15)
20/08/06 13:50:31 INFO Executor: Finished task 14.0 in stage 5.0 (TID 41). 786 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Finished task 14.0 in stage 5.0 (TID 41) in 19 ms on localhost (executor driver) (14/15)
20/08/06 13:50:31 INFO Executor: Finished task 13.0 in stage 5.0 (TID 40). 3530 bytes result sent to driver
20/08/06 13:50:31 INFO TaskSetManager: Finished task 13.0 in stage 5.0 (TID 40) in 27 ms on localhost (executor driver) (15/15)
20/08/06 13:50:31 INFO TaskSchedulerImpl: Removed TaskSet 5.0, whose tasks have all completed, from pool 
20/08/06 13:50:31 INFO DAGScheduler: ResultStage 5 (first at Feature.scala:119) finished in 0.086 s
20/08/06 13:50:31 INFO DAGScheduler: Job 5 finished: first at Feature.scala:119, took 0.090081 s
20/08/06 13:50:31 INFO MemoryStore: Block broadcast_9 stored as values in memory (estimated size 8.4 KB, free 2004.1 MB)
20/08/06 13:50:31 INFO MemoryStore: Block broadcast_9_piece0 stored as bytes in memory (estimated size 900.0 B, free 2004.1 MB)
20/08/06 13:50:31 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on 56f1369e0bb4:43539 (size: 900.0 B, free: 2004.5 MB)
20/08/06 13:50:31 INFO SparkContext: Created broadcast 9 from broadcast at Feature.scala:86
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGILL (0x4) at pc=0x00007fb258ecfda9, pid=66, tid=0x00007fb30d283700
#
# JRE version: OpenJDK Runtime Environment (8.0_212-b04) (build 1.8.0_212-b04)
# Java VM: OpenJDK 64-Bit Server VM (25.212-b04 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libtensorflow_framework.so.1+0x744da9]  _GLOBAL__sub_I_loader.cc+0x99
#
# Core dump written. Default location: //core or core.66
#
# An error report file with more information is saved as:
# //hs_err_pid66.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
miloradtrninic commented 4 years ago

Any news?

maziyarpanahi commented 4 years ago

We are still working on reproducing TF related issues. We check the Windows 8 and Windows 10 and today or tomorrow will get to this. Just to be 100% sure we are in the same environment as yours, would you mind adding any config/env, docker-compose commands, etc. to your repo so we can just follow those steps and have the exact same thing you have?

maziyarpanahi commented 4 years ago

PS: Our guess is a conflict between linking TensorFlow and something else similar to this issue: https://github.com/tensorflow/tensorflow/issues/24976 But we would like to have the exact similar environment to be sure about everything inside.

miloradtrninic commented 4 years ago

Here are the Ubuntu 18 based images https://github.com/miloradtrninic/docker-spark-images Just run build-images.sh Run docker compose from https://github.com/miloradtrninic/entity Just change image names to aggregation/spark-(master/worker):2.4.0-hadoop2.7

maziyarpanahi commented 4 years ago

@miloradtrninic Thanks for the instructions. We noticed ../envs/hadoop.env in the docker file, I think we need that in order to have the cluster correctly up and running?

miloradtrninic commented 4 years ago

Added envs

cgpeter96 commented 3 years ago

Any one solve this problem? @maziyarpanahi @manangandhi7 I think it is the error of libtensorflow.so

maziyarpanahi commented 3 years ago

Any one solve this problem? @maziyarpanahi @manangandhi7 I think it is the error of libtensorflow.so

It is related to the compatibility of libtensorflow.so on the OS and the related dependencies. Nothing can be done from inside the Spark NLP itself since it's related to TensorFlow 1.15 Java, but please do create a new issue with all the info required for us to reproduce. Maybe it can be solved easily if we can reproduce it.