search

dotnet / spark

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
https://dot.net/spark
MIT License
2.02k stars 315 forks source link

[BUG]: Issue with running Spark. NET app on Azure Databricks - SparkStoppedException #126

Closed AAATechGuy closed 5 years ago

AAATechGuy commented 5 years ago

Describe the bug SparkStoppedException while following instructions to run Spark .NET app on databricks (Azure).

To Reproduce

Steps to reproduce the behavior:

image

image

Errors...

19/05/31 05:15:52 INFO DotnetRunner: Starting DotnetBackend with /databricks/driver/HelloSpark/HelloSpark.
19/05/31 05:15:52 INFO DotnetRunner: Port number used by DotnetBackend is 35433
19/05/31 05:15:52 INFO DotnetRunner: Adding key=spark.executor.extraJavaOptions and value=-Djava.io.tmpdir=/local_disk0/tmp -XX:ReservedCodeCacheSize=256m -XX:+UseCodeCacheFlushing -Djava.security.properties=/databricks/spark/dbconf/java/extra.security -XX:+PrintFlagsFinal -XX:+PrintGCDateStamps -verbose:gc -XX:+PrintGCDetails -Xss4m -Djavax.xml.datatype.DatatypeFactory=com.sun.org.apache.xerces.internal.jaxp.datatype.DatatypeFactoryImpl -Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl -Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl -Djavax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema=com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchemaFactory -Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser -Dorg.w3c.dom.DOMImplementationSourceList=com.sun.org.apache.xerces.internal.dom.DOMXSImplementationSourceImpl -Ddatabricks.serviceName=spark-executor-1 to environment
19/05/31 05:15:52 INFO DotnetRunner: Adding key=spark.ui.port and value=44680 to environment
19/05/31 05:15:52 INFO DotnetRunner: Adding key=spark.executor.memory and value=7284m to environment
19/05/31 05:15:52 INFO DotnetRunner: Adding key=spark.executor.extraClassPath and value=/databricks/spark/dbconf/log4j/executor:/databricks/spark/dbconf/jets3t/:/databricks/spark/dbconf/hadoop:/databricks/hive/conf:/databricks/jars/api-base--api-base_java-spark_2.4_2.11_deploy.jar:/databricks/jars/api-base--api-base-spark_2.4_2.11_deploy.jar:/databricks/jars/api--rpc--rpc_parser-spark_2.4_2.11_deploy.jar:/databricks/jars/chauffeur-api--api--endpoints--endpoints-spark_2.4_2.11_deploy.jar:/databricks/jars/chauffeur-api--chauffeur-api-spark_2.4_2.11_deploy.jar:/databricks/jars/common--client--client-spark_2.4_2.11_deploy.jar:/databricks/jars/common--common-spark_2.4_2.11_deploy.jar:/databricks/jars/common--credentials--credentials-spark_2.4_2.11_deploy.jar:/databricks/jars/common--hadoop--hadoop-spark_2.4_2.11_deploy.jar:/databricks/jars/common--jetty--client--client-spark_2.4_2.11_deploy.jar:/databricks/jars/common--lazy--lazy-spark_2.4_2.11_deploy.jar:/databricks/jars/common--libcommon_resources.jar:/databricks/jars/common--path--path-spark_2.4_2.11_deploy.jar:/databricks/jars/common--rate-limiter--rate-limiter-spark_2.4_2.11_deploy.jar:/databricks/jars/common--storage--storage-spark_2.4_2.11_deploy.jar:/databricks/jars/daemon--data--client--client-spark_2.4_2.11_deploy.jar:/databricks/jars/daemon--data--client--conf--conf-spark_2.4_2.11_deploy.jar:/databricks/jars/daemon--data--client--utils-spark_2.4_2.11_deploy.jar:/databricks/jars/daemon--data--data-common--data-common-spark_2.4_2.11_deploy.jar:/databricks/jars/dbfs--utils--dbfs-utils-spark_2.4_2.11_deploy.jar:/databricks/jars/extern--acl--auth--auth-spark_2.4_2.11_deploy.jar:/databricks/jars/extern--extern-spark_2.4_2.11_deploy.jar:/databricks/jars/extern--libaws-regions.jar:/databricks/jars/----jackson_annotations_shaded--libjackson-annotations.jar:/databricks/jars/----jackson_core_shaded--libjackson-core.jar:/databricks/jars/----jackson_databind_shaded--libjackson-databind.jar:/databricks/jars/----jackson_datatype_joda_shaded--libjackson-datatype-joda.jar:/databricks/jars/jsonutil--jsonutil-spark_2.4_2.11_deploy.jar:/databricks/jars/logging--log4j-mod--log4j-mod-spark_2.4_2.11_deploy.jar:/databricks/jars/logging--utils--logging-utils-spark_2.4_2.11_deploy.jar:/databricks/jars/s3commit--client--client-spark_2.4_2.11_deploy.jar:/databricks/jars/s3commit--common--common-spark_2.4_2.11_deploy.jar:/databricks/jars/s3--s3-spark_2.4_2.11_deploy.jar:/databricks/jars/secret-manager--api--api-spark_2.4_2.11_deploy.jar:/databricks/jars/spark--common--spark-common-spark_2.4_2.11_deploy.jar:/databricks/jars/spark--conf-reader--conf-reader_lib-spark_2.4_2.11_deploy.jar:/databricks/jars/spark--dbutils--dbutils-api-spark_2.4_2.11_deploy.jar:/databricks/jars/spark--driver--antlr--parser-spark_2.4_2.11_deploy.jar:/databricks/jars/spark--driver--common--driver-common-spark_2.4_2.11_deploy.jar:/databricks/jars/spark--driver--display--display-spark_2.4_2.11_deploy.jar:/databricks/jars/spark--driver--driver-spark_2.4_2.11_deploy.jar:/databricks/jars/spark--driver--events-spark_2.4_2.11_deploy.jar:/databricks/jars/spark--driver--ml--ml-spark_2.4_2.11_deploy.jar:/databricks/jars/spark--driver--secret-redaction-spark_2.4_2.11_deploy.jar:/databricks/jars/spark--driver--spark--resources-resources.jar:/databricks/jars/spark--maven-trees--spark_2.4--antlr--antlr--antlr__antlr__2.7.7.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--amazon-kinesis-client--com.amazonaws__amazon-kinesis-client__1.8.10.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-autoscaling--com.amazonaws__aws-java-sdk-autoscaling__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-cloudformation--com.amazonaws__aws-java-sdk-cloudformation__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-cloudfront--com.amazonaws__aws-java-sdk-cloudfront__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-cloudhsm--com.amazonaws__aws-java-sdk-cloudhsm__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-cloudsearch--com.amazonaws__aws-java-sdk-cloudsearch__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-cloudtrail--com.amazonaws__aws-java-sdk-cloudtrail__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-cloudwatch--com.amazonaws__aws-java-sdk-cloudwatch__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-cloudwatchmetrics--com.amazonaws__aws-java-sdk-cloudwatchmetrics__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-codedeploy--com.amazonaws__aws-java-sdk-codedeploy__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-cognitoidentity--com.amazonaws__aws-java-sdk-cognitoidentity__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-cognitosync--com.amazonaws__aws-java-sdk-cognitosync__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-config--com.amazonaws__aws-java-sdk-config__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-core--com.amazonaws__aws-java-sdk-core__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-datapipeline--com.amazonaws__aws-java-sdk-datapipeline__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-directconnect--com.amazonaws__aws-java-sdk-directconnect__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-directory--com.amazonaws__aws-java-sdk-directory__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-dynamodb--com.amazonaws__aws-java-sdk-dynamodb__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-ec2--com.amazonaws__aws-java-sdk-ec2__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-ecs--com.amazonaws__aws-java-sdk-ecs__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-efs--com.amazonaws__aws-java-sdk-efs__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-elasticache--com.amazonaws__aws-java-sdk-elasticache__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-elasticbeanstalk--com.amazonaws__aws-java-sdk-elasticbeanstalk__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-elasticloadbalancing--com.amazonaws__aws-java-sdk-elasticloadbalancing__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-elastictranscoder--com.amazonaws__aws-java-sdk-elastictranscoder__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-emr--com.amazonaws__aws-java-sdk-emr__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-glacier--com.amazonaws__aws-java-sdk-glacier__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-iam--com.amazonaws__aws-java-sdk-iam__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-importexport--com.amazonaws__aws-java-sdk-importexport__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-kinesis--com.amazonaws__aws-java-sdk-kinesis__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-kms--com.amazonaws__aws-java-sdk-kms__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-lambda--com.amazonaws__aws-java-sdk-lambda__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-logs--com.amazonaws__aws-java-sdk-logs__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-machinelearning--com.amazonaws__aws-java-sdk-machinelearning__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-opsworks--com.amazonaws__aws-java-sdk-opsworks__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-rds--com.amazonaws__aws-java-sdk-rds__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-redshift--com.amazonaws__aws-java-sdk-redshift__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-route53--com.amazonaws__aws-java-sdk-route53__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-s3--com.amazonaws__aws-java-sdk-s3__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-ses--com.amazonaws__aws-java-sdk-ses__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-simpledb--com.amazonaws__aws-java-sdk-simpledb__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-simpleworkflow--com.amazonaws__aws-java-sdk-simpleworkflow__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-sns--com.amazonaws__aws-java-sdk-sns__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-sqs--com.amazonaws__aws-java-sdk-sqs__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-ssm--com.amazonaws__aws-java-sdk-ssm__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-storagegateway--com.amazonaws__aws-java-sdk-storagegateway__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-sts--com.amazonaws__aws-java-sdk-sts__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-support--com.amazonaws__aws-java-sdk-support__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-swf-libraries--com.amazonaws__aws-java-sdk-swf-libraries__1.11.22.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--aws-java-sdk-workspaces--com.amazonaws__aws-java-sdk-workspaces__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.amazonaws--jmespath-java--com.amazonaws__jmespath-java__1.11.313.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.carrotsearch--hppc--com.carrotsearch__hppc__0.7.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.chuusai--shapeless_2.11--com.chuusai__shapeless_2.11__2.3.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.clearspring.analytics--stream--com.clearspring.analytics__stream__2.7.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.databricks--dbml-local_2.11--com.databricks__dbml-local_2.11__0.5.0-db8-spark2.4.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.databricks--dbml-local_2.11-tests--com.databricks__dbml-local_2.11-tests__0.5.0-db8-spark2.4.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.databricks--jets3t--com.databricks__jets3t__0.7.1-0.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.databricks--Rserve--com.databricks__Rserve__1.8-3.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.databricks.scalapb--compilerplugin_2.11--com.databricks.scalapb__compilerplugin_2.11__0.4.15-9.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.databricks.scalapb--scalapb-runtime_2.11--com.databricks.scalapb__scalapb-runtime_2.11__0.4.15-9.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.esotericsoftware--kryo-shaded--com.esotericsoftware__kryo-shaded__4.0.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.esotericsoftware--minlog--com.esotericsoftware__minlog__1.3.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.fasterxml--classmate--com.fasterxml__classmate__1.0.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.fasterxml.jackson.core--jackson-annotations--com.fasterxml.jackson.core__jackson-annotations__2.6.7.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.fasterxml.jackson.core--jackson-core--com.fasterxml.jackson.core__jackson-core__2.6.7.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.fasterxml.jackson.core--jackson-databind--com.fasterxml.jackson.core__jackson-databind__2.6.7.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.fasterxml.jackson.dataformat--jackson-dataformat-cbor--com.fasterxml.jackson.dataformat__jackson-dataformat-cbor__2.6.7.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.fasterxml.jackson.datatype--jackson-datatype-joda--com.fasterxml.jackson.datatype__jackson-datatype-joda__2.6.7.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.fasterxml.jackson.module--jackson-module-paranamer--com.fasterxml.jackson.module__jackson-module-paranamer__2.6.7.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.fasterxml.jackson.module--jackson-module-scala_2.11--com.fasterxml.jackson.module__jackson-module-scala_2.11__2.6.7.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.github.fommil--jniloader--com.github.fommil__jniloader__1.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.github.fommil.netlib--core--com.github.fommil.netlib__core__1.1.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.github.fommil.netlib--native_ref-java--com.github.fommil.netlib__native_ref-java__1.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.github.fommil.netlib--native_ref-java-natives--com.github.fommil.netlib__native_ref-java-natives__1.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.github.fommil.netlib--native_system-java--com.github.fommil.netlib__native_system-java__1.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.github.fommil.netlib--native_system-java-natives--com.github.fommil.netlib__native_system-java-natives__1.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.github.fommil.netlib--netlib-native_ref-linux-x86_64-natives--com.github.fommil.netlib__netlib-native_ref-linux-x86_64-natives__1.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.github.fommil.netlib--netlib-native_system-linux-x86_64-natives--com.github.fommil.netlib__netlib-native_system-linux-x86_64-natives__1.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.github.luben--zstd-jni--com.github.luben__zstd-jni__1.3.2-2.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.github.rwl--jtransforms--com.github.rwl__jtransforms__2.4.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.google.code.findbugs--jsr305--com.google.code.findbugs__jsr305__2.0.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.google.code.gson--gson--com.google.code.gson__gson__2.2.4.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.googlecode.javaewah--JavaEWAH--com.googlecode.javaewah__JavaEWAH__0.3.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.google.guava--guava--com.google.guava__guava__15.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.google.protobuf--protobuf-java--com.google.protobuf__protobuf-java__2.6.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.h2database--h2--com.h2database__h2__1.3.174.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.jcraft--jsch--com.jcraft__jsch__0.1.50.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.jolbox--bonecp--com.jolbox__bonecp__0.8.0.RELEASE.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.microsoft.azure--azure-data-lake-store-sdk--com.microsoft.azure__azure-data-lake-store-sdk__2.2.8.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.microsoft.sqlserver--mssql-jdbc--com.microsoft.sqlserver__mssql-jdbc__6.2.2.jre8.jar:/databricks/jars/spark--maven-trees--spark_2.4--commons-beanutils--commons-beanutils--commons-beanutils__commons-beanutils__1.7.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--commons-beanutils--commons-beanutils-core--commons-beanutils__commons-beanutils-core__1.8.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--commons-cli--commons-cli--commons-cli__commons-cli__1.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--commons-codec--commons-codec--commons-codec__commons-codec__1.10.jar:/databricks/jars/spark--maven-trees--spark_2.4--commons-collections--commons-collections--commons-collections__commons-collections__3.2.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--commons-configuration--commons-configuration--commons-configuration__commons-configuration__1.6.jar:/databricks/jars/spark--maven-trees--spark_2.4--commons-dbcp--commons-dbcp--commons-dbcp__commons-dbcp__1.4.jar:/databricks/jars/spark--maven-trees--spark_2.4--commons-digester--commons-digester--commons-digester__commons-digester__1.8.jar:/databricks/jars/spark--maven-trees--spark_2.4--commons-httpclient--commons-httpclient--commons-httpclient__commons-httpclient__3.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--commons-io--commons-io--commons-io__commons-io__2.4.jar:/databricks/jars/spark--maven-trees--spark_2.4--commons-lang--commons-lang--commons-lang__commons-lang__2.6.jar:/databricks/jars/spark--maven-trees--spark_2.4--commons-logging--commons-logging--commons-logging__commons-logging__1.1.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--commons-net--commons-net--commons-net__commons-net__3.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--commons-pool--commons-pool--commons-pool__commons-pool__1.5.4.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.ning--compress-lzf--com.ning__compress-lzf__1.0.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.sun.mail--javax.mail--com.sun.mail__javax.mail__1.5.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.thoughtworks.paranamer--paranamer--com.thoughtworks.paranamer__paranamer__2.8.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.trueaccord.lenses--lenses_2.11--com.trueaccord.lenses__lenses_2.11__0.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.twitter--chill_2.11--com.twitter__chill_2.11__0.9.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.twitter--chill-java--com.twitter__chill-java__0.9.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.twitter--parquet-hadoop-bundle--com.twitter__parquet-hadoop-bundle__1.6.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.twitter--util-app_2.11--com.twitter__util-app_2.11__6.23.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.twitter--util-core_2.11--com.twitter__util-core_2.11__6.23.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.twitter--util-jvm_2.11--com.twitter__util-jvm_2.11__6.23.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.typesafe--config--com.typesafe__config__1.2.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.typesafe.scala-logging--scala-logging-api_2.11--com.typesafe.scala-logging__scala-logging-api_2.11__2.1.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.typesafe.scala-logging--scala-logging-slf4j_2.11--com.typesafe.scala-logging__scala-logging-slf4j_2.11__2.1.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.univocity--univocity-parsers--com.univocity__univocity-parsers__2.7.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.vlkan--flatbuffers--com.vlkan__flatbuffers__1.2.0-3f79e055.jar:/databricks/jars/spark--maven-trees--spark_2.4--com.zaxxer--HikariCP--com.zaxxer__HikariCP__3.1.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--info.ganglia.gmetric4j--gmetric4j--info.ganglia.gmetric4j__gmetric4j__1.0.7.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.airlift--aircompressor--io.airlift__aircompressor__0.10.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.dropwizard.metrics--metrics-core--io.dropwizard.metrics__metrics-core__3.1.5.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.dropwizard.metrics--metrics-ganglia--io.dropwizard.metrics__metrics-ganglia__3.1.5.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.dropwizard.metrics--metrics-graphite--io.dropwizard.metrics__metrics-graphite__3.1.5.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.dropwizard.metrics--metrics-healthchecks--io.dropwizard.metrics__metrics-healthchecks__3.1.5.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.dropwizard.metrics--metrics-jetty9--io.dropwizard.metrics__metrics-jetty9__3.1.5.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.dropwizard.metrics--metrics-json--io.dropwizard.metrics__metrics-json__3.1.5.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.dropwizard.metrics--metrics-jvm--io.dropwizard.metrics__metrics-jvm__3.1.5.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.dropwizard.metrics--metrics-log4j--io.dropwizard.metrics__metrics-log4j__3.1.5.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.dropwizard.metrics--metrics-servlets--io.dropwizard.metrics__metrics-servlets__3.1.5.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.netty--netty-all--io.netty__netty-all__4.1.17.Final.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.netty--netty--io.netty__netty__3.9.9.Final.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.prometheus.jmx--collector--io.prometheus.jmx__collector__0.7.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.prometheus--simpleclient_common--io.prometheus__simpleclient_common__0.0.16.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.prometheus--simpleclient_dropwizard--io.prometheus__simpleclient_dropwizard__0.0.16.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.prometheus--simpleclient--io.prometheus__simpleclient__0.0.16.jar:/databricks/jars/spark--maven-trees--spark_2.4--io.prometheus--simpleclient_servlet--io.prometheus__simpleclient_servlet__0.0.16.jar:/databricks/jars/spark--maven-trees--spark_2.4--javax.activation--activation--javax.activation__activation__1.1.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--javax.annotation--javax.annotation-api--javax.annotation__javax.annotation-api__1.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--javax.el--javax.el-api--javax.el__javax.el-api__2.2.4.jar:/databricks/jars/spark--maven-trees--spark_2.4--javax.jdo--jdo-api--javax.jdo__jdo-api__3.0.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--javax.servlet--javax.servlet-api--javax.servlet__javax.servlet-api__3.1.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--javax.servlet.jsp--jsp-api--javax.servlet.jsp__jsp-api__2.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--javax.transaction--jta--javax.transaction__jta__1.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--javax.validation--validation-api--javax.validation__validation-api__1.1.0.Final.jar:/databricks/jars/spark--maven-trees--spark_2.4--javax.ws.rs--javax.ws.rs-api--javax.ws.rs__javax.ws.rs-api__2.0.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--javax.xml.bind--jaxb-api--javax.xml.bind__jaxb-api__2.2.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--javax.xml.stream--stax-api--javax.xml.stream__stax-api__1.0-2.jar:/databricks/jars/spark--maven-trees--spark_2.4--javolution--javolution--javolution__javolution__5.5.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--jline--jline--jline__jline__2.14.6.jar:/databricks/jars/spark--maven-trees--spark_2.4--joda-time--joda-time--joda-time__joda-time__2.9.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--log4j--apache-log4j-extras--log4j__apache-log4j-extras__1.2.17.jar:/databricks/jars/spark--maven-trees--spark_2.4--log4j--log4j--log4j__log4j__1.2.17.jar:/databricks/jars/spark--maven-trees--spark_2.4--net.hydromatic--eigenbase-properties--net.hydromatic__eigenbase-properties__1.1.5.jar:/databricks/jars/spark--maven-trees--spark_2.4--net.razorvine--pyrolite--net.razorvine__pyrolite__4.13.jar:/databricks/jars/spark--maven-trees--spark_2.4--net.sf.jpam--jpam--net.sf.jpam__jpam__1.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--net.sf.opencsv--opencsv--net.sf.opencsv__opencsv__2.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--net.sf.supercsv--super-csv--net.sf.supercsv__super-csv__2.2.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--net.snowflake--snowflake-ingest-sdk--net.snowflake__snowflake-ingest-sdk__0.9.5.jar:/databricks/jars/spark--maven-trees--spark_2.4--net.snowflake--snowflake-jdbc--net.snowflake__snowflake-jdbc__3.6.15.jar:/databricks/jars/spark--maven-trees--spark_2.4--net.snowflake--spark-snowflake_2.11--net.snowflake__spark-snowflake_2.11__2.4.10-spark_2.4.jar:/databricks/jars/spark--maven-trees--spark_2.4--net.sourceforge.f2j--arpack_combined_all--net.sourceforge.f2j__arpack_combined_all__0.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.acplt--oncrpc--org.acplt__oncrpc__1.0.7.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.antlr--antlr4-runtime--org.antlr__antlr4-runtime__4.7.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.antlr--antlr-runtime--org.antlr__antlr-runtime__3.4.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.antlr--ST4--org.antlr__ST4__4.0.4.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.antlr--stringtemplate--org.antlr__stringtemplate__3.2.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.ant--ant-jsch--org.apache.ant__ant-jsch__1.9.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.ant--ant-launcher--org.apache.ant__ant-launcher__1.9.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.ant--ant--org.apache.ant__ant__1.9.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.arrow--arrow-format--org.apache.arrow__arrow-format__0.10.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.arrow--arrow-memory--org.apache.arrow__arrow-memory__0.10.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.arrow--arrow-vector--org.apache.arrow__arrow-vector__0.10.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.avro--avro-ipc--org.apache.avro__avro-ipc__1.8.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.avro--avro-mapred-hadoop2--org.apache.avro__avro-mapred-hadoop2__1.8.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.avro--avro--org.apache.avro__avro__1.8.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.calcite--calcite-avatica--org.apache.calcite__calcite-avatica__1.2.0-incubating.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.calcite--calcite-core--org.apache.calcite__calcite-core__1.2.0-incubating.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.calcite--calcite-linq4j--org.apache.calcite__calcite-linq4j__1.2.0-incubating.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.commons--commons-compress--org.apache.commons__commons-compress__1.8.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.commons--commons-crypto--org.apache.commons__commons-crypto__1.0.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.commons--commons-lang3--org.apache.commons__commons-lang3__3.5.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.commons--commons-math3--org.apache.commons__commons-math3__3.4.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.curator--curator-client--org.apache.curator__curator-client__2.7.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.curator--curator-framework--org.apache.curator__curator-framework__2.7.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.curator--curator-recipes--org.apache.curator__curator-recipes__2.7.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.derby--derby--org.apache.derby__derby__10.12.1.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.directory.api--api-asn1-api--org.apache.directory.api__api-asn1-api__1.0.0-M20.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.directory.api--api-util--org.apache.directory.api__api-util__1.0.0-M20.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.directory.server--apacheds-i18n--org.apache.directory.server__apacheds-i18n__2.0.0-M15.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.directory.server--apacheds-kerberos-codec--org.apache.directory.server__apacheds-kerberos-codec__2.0.0-M15.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.hadoop--hadoop-annotations--org.apache.hadoop__hadoop-annotations__2.7.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.hadoop--hadoop-auth--org.apache.hadoop__hadoop-auth__2.7.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.hadoop--hadoop-client--org.apache.hadoop__hadoop-client__2.7.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.hadoop--hadoop-common--org.apache.hadoop__hadoop-common__2.7.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.hadoop--hadoop-hdfs--org.apache.hadoop__hadoop-hdfs__2.7.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.hadoop--hadoop-mapreduce-client-app--org.apache.hadoop__hadoop-mapreduce-client-app__2.7.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.hadoop--hadoop-mapreduce-client-common--org.apache.hadoop__hadoop-mapreduce-client-common__2.7.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.hadoop--hadoop-mapreduce-client-core--org.apache.hadoop__hadoop-mapreduce-client-core__2.7.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.hadoop--hadoop-mapreduce-client-jobclient--org.apache.hadoop__hadoop-mapreduce-client-jobclient__2.7.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.hadoop--hadoop-mapreduce-client-shuffle--org.apache.hadoop__hadoop-mapreduce-client-shuffle__2.7.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.hadoop--hadoop-yarn-api--org.apache.hadoop__hadoop-yarn-api__2.7.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.hadoop--hadoop-yarn-client--org.apache.hadoop__hadoop-yarn-client__2.7.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.hadoop--hadoop-yarn-common--org.apache.hadoop__hadoop-yarn-common__2.7.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.hadoop--hadoop-yarn-server-common--org.apache.hadoop__hadoop-yarn-server-common__2.7.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.htrace--htrace-core--org.apache.htrace__htrace-core__3.1.0-incubating.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.httpcomponents--httpclient--org.apache.httpcomponents__httpclient__4.5.4.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.httpcomponents--httpcore--org.apache.httpcomponents__httpcore__4.4.8.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.ivy--ivy--org.apache.ivy__ivy__2.4.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.orc--orc-core-nohive--org.apache.orc__orc-core-nohive__1.5.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.orc--orc-mapreduce-nohive--org.apache.orc__orc-mapreduce-nohive__1.5.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.orc--orc-shims--org.apache.orc__orc-shims__1.5.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.parquet--parquet-column--org.apache.parquet__parquet-column__1.10.1.1-databricks3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.parquet--parquet-common--org.apache.parquet__parquet-common__1.10.1.1-databricks3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.parquet--parquet-encoding--org.apache.parquet__parquet-encoding__1.10.1.1-databricks3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.parquet--parquet-format--org.apache.parquet__parquet-format__2.4.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.parquet--parquet-hadoop--org.apache.parquet__parquet-hadoop__1.10.1.1-databricks3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.parquet--parquet-jackson--org.apache.parquet__parquet-jackson__1.10.1.1-databricks3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.thrift--libfb303--org.apache.thrift__libfb303__0.9.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.thrift--libthrift--org.apache.thrift__libthrift__0.9.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.xbean--xbean-asm6-shaded--org.apache.xbean__xbean-asm6-shaded__4.8.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.apache.zookeeper--zookeeper--org.apache.zookeeper__zookeeper__3.4.6.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.codehaus.jackson--jackson-core-asl--org.codehaus.jackson__jackson-core-asl__1.9.13.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.codehaus.jackson--jackson-jaxrs--org.codehaus.jackson__jackson-jaxrs__1.9.13.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.codehaus.jackson--jackson-mapper-asl--org.codehaus.jackson__jackson-mapper-asl__1.9.13.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.codehaus.jackson--jackson-xc--org.codehaus.jackson__jackson-xc__1.9.13.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.codehaus.janino--commons-compiler--org.codehaus.janino__commons-compiler__3.0.10.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.codehaus.janino--janino--org.codehaus.janino__janino__3.0.10.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.datanucleus--datanucleus-api-jdo--org.datanucleus__datanucleus-api-jdo__3.2.6.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.datanucleus--datanucleus-core--org.datanucleus__datanucleus-core__3.2.10.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.datanucleus--datanucleus-rdbms--org.datanucleus__datanucleus-rdbms__3.2.9.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.eclipse.jetty--jetty-client--org.eclipse.jetty__jetty-client__9.3.20.v20170531.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.eclipse.jetty--jetty-continuation--org.eclipse.jetty__jetty-continuation__9.3.20.v20170531.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.eclipse.jetty--jetty-http--org.eclipse.jetty__jetty-http__9.3.20.v20170531.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.eclipse.jetty--jetty-io--org.eclipse.jetty__jetty-io__9.3.20.v20170531.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.eclipse.jetty--jetty-jndi--org.eclipse.jetty__jetty-jndi__9.3.20.v20170531.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.eclipse.jetty--jetty-plus--org.eclipse.jetty__jetty-plus__9.3.20.v20170531.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.eclipse.jetty--jetty-proxy--org.eclipse.jetty__jetty-proxy__9.3.20.v20170531.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.eclipse.jetty--jetty-security--org.eclipse.jetty__jetty-security__9.3.20.v20170531.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.eclipse.jetty--jetty-server--org.eclipse.jetty__jetty-server__9.3.20.v20170531.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.eclipse.jetty--jetty-servlet--org.eclipse.jetty__jetty-servlet__9.3.20.v20170531.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.eclipse.jetty--jetty-servlets--org.eclipse.jetty__jetty-servlets__9.3.20.v20170531.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.eclipse.jetty--jetty-util--org.eclipse.jetty__jetty-util__9.3.20.v20170531.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.eclipse.jetty--jetty-webapp--org.eclipse.jetty__jetty-webapp__9.3.20.v20170531.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.eclipse.jetty--jetty-xml--org.eclipse.jetty__jetty-xml__9.3.20.v20170531.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.fusesource.leveldbjni--leveldbjni-all--org.fusesource.leveldbjni__leveldbjni-all__1.8.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.glassfish.hk2.external--aopalliance-repackaged--org.glassfish.hk2.external__aopalliance-repackaged__2.4.0-b34.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.glassfish.hk2.external--javax.inject--org.glassfish.hk2.external__javax.inject__2.4.0-b34.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.glassfish.hk2--hk2-api--org.glassfish.hk2__hk2-api__2.4.0-b34.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.glassfish.hk2--hk2-locator--org.glassfish.hk2__hk2-locator__2.4.0-b34.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.glassfish.hk2--hk2-utils--org.glassfish.hk2__hk2-utils__2.4.0-b34.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.glassfish.hk2--osgi-resource-locator--org.glassfish.hk2__osgi-resource-locator__1.0.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.glassfish.jersey.bundles.repackaged--jersey-guava--org.glassfish.jersey.bundles.repackaged__jersey-guava__2.22.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.glassfish.jersey.containers--jersey-container-servlet-core--org.glassfish.jersey.containers__jersey-container-servlet-core__2.22.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.glassfish.jersey.containers--jersey-container-servlet--org.glassfish.jersey.containers__jersey-container-servlet__2.22.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.glassfish.jersey.core--jersey-client--org.glassfish.jersey.core__jersey-client__2.22.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.glassfish.jersey.core--jersey-common--org.glassfish.jersey.core__jersey-common__2.22.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.glassfish.jersey.core--jersey-server--org.glassfish.jersey.core__jersey-server__2.22.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.glassfish.jersey.media--jersey-media-jaxb--org.glassfish.jersey.media__jersey-media-jaxb__2.22.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.hibernate--hibernate-validator--org.hibernate__hibernate-validator__5.1.1.Final.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.iq80.snappy--snappy--org.iq80.snappy__snappy__0.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.javassist--javassist--org.javassist__javassist__3.18.1-GA.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.jboss.logging--jboss-logging--org.jboss.logging__jboss-logging__3.1.3.GA.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.jdbi--jdbi--org.jdbi__jdbi__2.63.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.joda--joda-convert--org.joda__joda-convert__1.7.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.jodd--jodd-core--org.jodd__jodd-core__3.5.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.json4s--json4s-ast_2.11--org.json4s__json4s-ast_2.11__3.5.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.json4s--json4s-core_2.11--org.json4s__json4s-core_2.11__3.5.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.json4s--json4s-jackson_2.11--org.json4s__json4s-jackson_2.11__3.5.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.json4s--json4s-scalap_2.11--org.json4s__json4s-scalap_2.11__3.5.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.lz4--lz4-java--org.lz4__lz4-java__1.4.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.mariadb.jdbc--mariadb-java-client--org.mariadb.jdbc__mariadb-java-client__2.1.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.mockito--mockito-all--org.mockito__mockito-all__1.9.5.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.objenesis--objenesis--org.objenesis__objenesis__2.5.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.postgresql--postgresql--org.postgresql__postgresql__42.1.4.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.roaringbitmap--RoaringBitmap--org.roaringbitmap__RoaringBitmap__0.5.11.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.rocksdb--rocksdbjni--org.rocksdb__rocksdbjni__5.2.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.rosuda.REngine--REngine--org.rosuda.REngine__REngine__2.1.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.scalacheck--scalacheck_2.11--org.scalacheck__scalacheck_2.11__1.12.5.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.scalactic--scalactic_2.11--org.scalactic__scalactic_2.11__3.0.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.scala-lang.modules--scala-parser-combinators_2.11--org.scala-lang.modules__scala-parser-combinators_2.11__1.1.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.scala-lang.modules--scala-xml_2.11--org.scala-lang.modules__scala-xml_2.11__1.0.5.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.scala-lang--scala-compiler_2.11--org.scala-lang__scala-compiler__2.11.12.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.scala-lang--scala-library_2.11--org.scala-lang__scala-library__2.11.12.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.scala-lang--scala-reflect_2.11--org.scala-lang__scala-reflect__2.11.12.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.scalanlp--breeze_2.11--org.scalanlp__breeze_2.11__0.13.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.scalanlp--breeze-macros_2.11--org.scalanlp__breeze-macros_2.11__0.13.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.scala-sbt--test-interface--org.scala-sbt__test-interface__1.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.scalatest--scalatest_2.11--org.scalatest__scalatest_2.11__3.0.3.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.slf4j--jcl-over-slf4j--org.slf4j__jcl-over-slf4j__1.7.16.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.slf4j--jul-to-slf4j--org.slf4j__jul-to-slf4j__1.7.16.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.slf4j--slf4j-api--org.slf4j__slf4j-api__1.7.16.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.slf4j--slf4j-log4j12--org.slf4j__slf4j-log4j12__1.7.16.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.spark-project.hive--hive-beeline--org.spark-project.hive__hive-beeline__1.2.1.spark2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.spark-project.hive--hive-cli--org.spark-project.hive__hive-cli__1.2.1.spark2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.spark-project.hive--hive-exec--org.spark-project.hive__hive-exec__1.2.1.spark2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.spark-project.hive--hive-jdbc--org.spark-project.hive__hive-jdbc__1.2.1.spark2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.spark-project.hive--hive-metastore--org.spark-project.hive__hive-metastore__1.2.1.spark2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.spark-project.spark--unused--org.spark-project.spark__unused__1.0.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.spire-math--spire_2.11--org.spire-math__spire_2.11__0.13.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.spire-math--spire-macros_2.11--org.spire-math__spire-macros_2.11__0.13.0.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.springframework--spring-core--org.springframework__spring-core__4.1.4.RELEASE.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.springframework--spring-test--org.springframework__spring-test__4.1.4.RELEASE.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.tukaani--xz--org.tukaani__xz__1.5.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.typelevel--machinist_2.11--org.typelevel__machinist_2.11__0.6.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.typelevel--macro-compat_2.11--org.typelevel__macro-compat_2.11__1.1.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.xerial.snappy--snappy-java--org.xerial.snappy__snappy-java__1.1.7.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.xerial--sqlite-jdbc--org.xerial__sqlite-jdbc__3.8.11.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--org.yaml--snakeyaml--org.yaml__snakeyaml__1.16.jar:/databricks/jars/spark--maven-trees--spark_2.4--oro--oro--oro__oro__2.0.8.jar:/databricks/jars/spark--maven-trees--spark_2.4--software.amazon.ion--ion-java--software.amazon.ion__ion-java__1.0.2.jar:/databricks/jars/spark--maven-trees--spark_2.4--stax--stax-api--stax__stax-api__1.0.1.jar:/databricks/jars/spark--maven-trees--spark_2.4--xmlenc--xmlenc--xmlenc__xmlenc__0.52.jar:/databricks/jars/spark--sql-extension--sql-extension-spark_2.4_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--avro_2.11_deploy_shaded.jar:/databricks/jars/spark--versions--2.4--catalyst_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--com.fasterxml.jackson.core__jackson-annotations__2.9.0_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--com.fasterxml.jackson.core__jackson-core__2.9.4_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--com.fasterxml.jackson.core__jackson-databind__2.9.4_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--com.fasterxml.jackson.jaxrs__jackson-jaxrs-base__2.9.4_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--com.fasterxml.jackson.jaxrs__jackson-jaxrs-json-provider__2.9.4_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--com.fasterxml.jackson.module__jackson-module-jaxb-annotations__2.9.4_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--com.github.shyiko__mysql-binlog-connector-java__0.16.1_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--com.google.guava__guava__20.0_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--core_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--debezium-connector-mysql_2.11_deploy_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--ganglia-lgpl_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--graphx_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--hive_2.11_deploy_shaded.jar:/databricks/jars/spark--versions--2.4--hive-thriftserver_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--io.confluent__common-utils__4.0.0_shaded-for-avro.jar:/databricks/jars/spark--versions--2.4--io.confluent__kafka-schema-registry-client__4.0.0_shaded-for-avro.jar:/databricks/jars/spark--versions--2.4--io.debezium__debezium-core__0.8.3.Final_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--io.debezium__debezium-ddl-parser__0.8.3.Final_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--io.debezium__debezium-embedded__0.8.3.Final_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--javax.annotation__javax.annotation-api__1.2_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--javax.servlet__javax.servlet-api__3.1.0_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--javax.validation__validation-api__1.1.0.Final_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--javax.ws.rs__javax.ws.rs-api__2.0.1_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--kafka_2.11_only_kafka08_shaded.jar:/databricks/jars/spark--versions--2.4--kafka-clients_only_kafka08_shaded.jar:/databricks/jars/spark--versions--2.4--kafka-clients_only_shaded.jar:/databricks/jars/spark--versions--2.4--libspark-sql-parser-compiled.jar:/databricks/jars/spark--versions--2.4--metrics-core_only_kafka08_shaded.jar:/databricks/jars/spark--versions--2.4--mllib_2.11_deploy_shaded.jar:/databricks/jars/spark--versions--2.4--mllib-local_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--mysql-cdc_2.11_deploy_shaded.jar:/databricks/jars/spark--versions--2.4--net.sourceforge.argparse4j__argparse4j__0.7.0_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.apache.commons__commons-pool2__2.5.0_shaded-for-hive.jar:/databricks/jars/spark--versions--2.4--org.apache.kafka__connect-api__1.1.0_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.apache.kafka__connect-file__1.1.0_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.apache.kafka__connect-json__1.1.0_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.apache.kafka__connect-runtime__1.1.0_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.apache.kafka__connect-transforms__1.1.0_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.apache.kafka__kafka-clients__2.0.0_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.apache.kafka__kafka-log4j-appender__1.1.0_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.apache.kafka__kafka-tools__1.1.0_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.eclipse.jetty__jetty-client__9.2.24.v20180105_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.eclipse.jetty__jetty-continuation__9.2.24.v20180105_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.eclipse.jetty__jetty-http__9.2.24.v20180105_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.eclipse.jetty__jetty-io__9.2.24.v20180105_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.eclipse.jetty__jetty-security__9.2.24.v20180105_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.eclipse.jetty__jetty-server__9.2.24.v20180105_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.eclipse.jetty__jetty-servlet__9.2.24.v20180105_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.eclipse.jetty__jetty-servlets__9.2.24.v20180105_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.eclipse.jetty__jetty-util__9.2.24.v20180105_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.glassfish.hk2.external__aopalliance-repackaged__2.5.0-b32_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.glassfish.hk2.external__javax.inject__2.5.0-b32_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.glassfish.hk2__hk2-api__2.5.0-b32_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.glassfish.hk2__hk2-locator__2.5.0-b32_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.glassfish.hk2__hk2-utils__2.5.0-b32_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.glassfish.hk2__osgi-resource-locator__1.0.1_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.glassfish.jersey.bundles.repackaged__jersey-guava__2.25.1_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.glassfish.jersey.containers__jersey-container-servlet__2.25.1_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.glassfish.jersey.containers__jersey-container-servlet-core__2.25.1_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.glassfish.jersey.core__jersey-client__2.25.1_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.glassfish.jersey.core__jersey-common__2.25.1_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.glassfish.jersey.core__jersey-server__2.25.1_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.glassfish.jersey.media__jersey-media-jaxb__2.25.1_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.javassist__javassist__3.21.0-GA_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.jpmml__pmml-model__1.2.15_shaded-for-mllib.jar:/databricks/jars/spark--versions--2.4--org.jpmml__pmml-schema__1.2.15_shaded-for-mllib.jar:/databricks/jars/spark--versions--2.4--org.lz4__lz4-java__1.4.1_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.reflections__reflections__0.9.11_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--org.xerial.snappy__snappy-java__1.1.7.1_shaded-for-mysql-cdc.jar:/databricks/jars/spark--versions--2.4--py4j_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--redshift_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--repl_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--shim_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--spark_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--spark-2.4-avro-resources-jar-resources.jar:/databricks/jars/spark--versions--2.4--spark-2.4-core-resources-jar-resources.jar:/databricks/jars/spark--versions--2.4--spark-2.4-hive-resources-jar-resources.jar:/databricks/jars/spark--versions--2.4--spark-2.4-mllib-resources-jar-resources.jar:/databricks/jars/spark--versions--2.4--spark-2.4-mysql-cdc-resources-jar-resources.jar:/databricks/jars/spark--versions--2.4--spark-2.4-spark-sql-aws-resources-jar-resources.jar:/databricks/jars/spark--versions--2.4--spark-2.4-spark-sql-azure-resources-jar-resources.jar:/databricks/jars/spark--versions--2.4--spark-2.4-spark-sql-kafka-0-10-resources-jar-resources.jar:/databricks/jars/spark--versions--2.4--spark-2.4-spark-sql-kafka-0-8-resources-jar-resources.jar:/databricks/jars/spark--versions--2.4--spark-2.4-sql-resources-jar-resources.jar:/databricks/jars/spark--versions--2.4--spark-2.4-streaming-resources-jar-resources.jar:/databricks/jars/spark--versions--2.4--spark-sql-aws_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--spark-sql-azure_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--spark-sql-kafka-0-10_2.11_deploy_shaded.jar:/databricks/jars/spark--versions--2.4--spark-sql-kafka-0-8_2.11_deploy_shaded.jar:/databricks/jars/spark--versions--2.4--spark-sql-notification_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--sql_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--sqldw_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--streaming_2.11_deploy.jar:/databricks/jars/spark--versions--2.4--tags_2.11_deploy.jar:/databricks/jars/third_party--azure--com.fasterxml.jackson.core__jackson-core__2.7.2_shaded.jar:/databricks/jars/third_party--azure--com.microsoft.azure__azure-keyvault-core__1.0.0_shaded.jar:/databricks/jars/third_party--azure--com.microsoft.azure__azure-storage__5.2.0_shaded.jar:/databricks/jars/third_party--azure--org.apache.commons__commons-lang3__3.4_shaded.jar:/databricks/jars/third_party--datalake--datalake-spark_2.4_2.11_deploy.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--aopalliance__aopalliance__1.0_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.fasterxml.jackson.core__jackson-annotations__2.7.0_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.fasterxml.jackson.core__jackson-core__2.7.2_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.fasterxml.jackson.core__jackson-core__2.7.2_2.11_shaded_20180920_b33d810.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.fasterxml.jackson.core__jackson-databind__2.7.2_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.fasterxml.jackson.datatype__jackson-datatype-joda__2.7.2_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.google.code.findbugs__jsr305__1.3.9_2.11_shaded_20180920_b33d810.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.google.guava__guava__11.0.2_2.11_shaded_20180920_b33d810.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.google.guava__guava__16.0_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.google.inject__guice__3.0_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.microsoft.azure__azure-annotations__1.2.0_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.microsoft.azure__azure-keyvault-core__1.0.0_2.11_shaded_20180920_b33d810.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.microsoft.azure__azure-storage__7.0.0_2.11_shaded_20180920_b33d810.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.microsoft.rest__client-runtime__1.1.0_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--commons-codec__commons-codec__1.9_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--commons-codec__commons-codec__1.9_2.11_shaded_20180920_b33d810.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--commons-logging__commons-logging__1.2_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--commons-logging__commons-logging__1.2_2.11_shaded_20180920_b33d810.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.squareup.okhttp3__logging-interceptor__3.3.1_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.squareup.okhttp3__okhttp__3.3.1_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.squareup.okhttp3__okhttp-urlconnection__3.3.1_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.squareup.okio__okio__1.8.0_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.squareup.retrofit2__adapter-rxjava__2.1.0_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.squareup.retrofit2__converter-jackson__2.1.0_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.squareup.retrofit2__retrofit__2.1.0_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--com.sun.xml.bind__jaxb-impl__2.2.3-1_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--hadoop-azure-2.7.3-abfs-external-20180625_3682417-spark_2.4_2.11_deploy_shaded.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--hadoop-azure-2.7.3-abfs-external-20180920_b33d810-spark_2.4_2.11_deploy_shaded.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--io.netty__netty-all__4.0.52.Final_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--io.reactivex__rxjava__1.2.4_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--javax.activation__activation__1.1_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--javax.inject__javax.inject__1_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--javax.xml.bind__jaxb-api__2.2.2_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--javax.xml.stream__stax-api__1.0-2_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--joda-time__joda-time__2.4_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--org.apache.htrace__htrace-core__3.1.0-incubating_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--org.apache.httpcomponents__httpclient__4.5.2_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--org.apache.httpcomponents__httpclient__4.5.2_2.11_shaded_20180920_b33d810.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--org.apache.httpcomponents__httpcore__4.4.4_2.11_shaded_20180625_3682417.jar:/databricks/jars/third_party--hadoop-azure-2.7.3-abfs--org.apache.httpcomponents__httpcore__4.4.4_2.11_shaded_20180920_b33d810.jar:/databricks/jars/third_party--jackson--guava_only_shaded.jar:/databricks/jars/third_party--jackson--jackson-module-scala-shaded_2.11_deploy.jar:/databricks/jars/third_party--jackson--jsr305_only_shaded.jar:/databricks/jars/third_party--jackson--paranamer_only_shaded.jar:/databricks/jars/third_party--jetty8-shaded-client--databricks-patched-jetty-client-jar_shaded.jar:/databricks/jars/third_party--jetty8-shaded-client--databricks-patched-jetty-http-jar_shaded.jar:/databricks/jars/third_party--jetty8-shaded-client--jetty-io_shaded.jar:/databricks/jars/third_party--jetty8-shaded-client--jetty-jmx_shaded.jar:/databricks/jars/third_party--jetty8-shaded-client--jetty-util_shaded.jar:/databricks/jars/utils--process_utils-spark_2.4_2.11_deploy.jar:/databricks/jars/workflow--workflow-spark_2.4_2.11_deploy.jar to environment
19/05/31 05:15:53 WARN SparkSession$Builder: Using an existing SparkSession; some configuration may not take effect.
19/05/31 05:15:53 WARN FileStreamSink: Error while looking for metadata directory.

19/05/31 05:16:09 WARN ScalaDriverWrapper: Spark is detected to be down after running a command
19/05/31 05:16:09 WARN ScalaDriverWrapper: Fatal exception (spark down) in ReplId-35961-644b8-8dcb8-a
com.databricks.backend.common.rpc.SparkStoppedException: Spark down: 
    at com.databricks.backend.daemon.driver.DriverWrapper.getCommandOutputAndError(DriverWrapper.scala:485)

.ch.EPollSelectorImpl@5dbacc8e
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
    at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
    at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
    at java.lang.Thread.run(Thread.java:748)

rpc-client-1-1 id=69 state=RUNNABLE (running in native)
    at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
    at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
    at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
      - locked java.util.Collections$UnmodifiableSet@361b7f90
      - locked sun.nio.ch.EPollSelectorImpl@3ef20b8f
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
    at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
    at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
    at java.lang.Thread.run(Thread.java:748)
... ... ...
... ... ...
... ... ...
Thread-1 id=11 state=RUNNABLE
    at sun.management.ThreadImpl.dumpThreads0(Native Method)
    at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:454)
    at com.codahale.metrics.jvm.ThreadDump.dump(ThreadDump.java:30)
    at com.databricks.util.StackTraceReporter.reportAllStackTracesToStream(StackTraceReporter.scala:83)
    at com.databricks.util.StackTraceReporter$.reportAllStackTracesToStream(StackTraceReporter.scala:128)
    at com.databricks.DatabricksMain$$anonfun$initDatabricks$2.apply$mcV$sp(DatabricksMain.scala:133)
    at com.databricks.DatabricksMain$$anon$2.run(DatabricksMain.scala:61)

log4j:WARN No appenders could be found for logger (com.databricks.conf.trusted.ProjectConf$).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0

log4j-active.txt stderr.txt stdout.txt

imback82 commented 5 years ago

I believe you shouldn't stop SparkContext (spark.Stop()) when you are running a job through "Set Jar". More info here: https://docs.databricks.com/user-guide/jobs.html#use-the-shared-sparkcontext