Failed to run titanic example, got java.lang.AbstractMethodError

[TechpriestW]

Describe the bug Exception in thread "main" java.lang.AbstractMethodError

To Reproduce ./gradlew -q sparkSubmit -Dmain=com.salesforce.hw.OpTitanicSimple -Dargs="\ pwd/src/main/resources/TitanicDataset/TitanicPassengersTrainData.csv"

Expected behavior Successfully run this example

Logs or screenshots Exception in thread "main" java.lang.AbstractMethodError at org.apache.spark.ml.param.Params$class.$init$(params.scala:868) at com.salesforce.op.stages.impl.tuning.Splitter.(Splitter.scala:47) at com.salesforce.op.stages.impl.tuning.DataSplitter.(DataSplitter.scala:62) at com.salesforce.op.stages.impl.tuning.DataSplitter$.apply(DataSplitter.scala:51) at com.salesforce.op.stages.impl.classification.BinaryClassificationModelSelector$.withTrainValidationSplit$default$1(BinaryClassificationModelSelector.scala:211) at com.salesforce.hw.OpTitanicSimple$.main(OpTitanicSimple.scala:135) at com.salesforce.hw.OpTitanicSimple.main(OpTitanicSimple.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

FAILURE: Build failed with an exception.

• What went wrong: Execution failed for task ':sparkSubmit'.

  Process 'command '/home/user/spark_workspace/spark-project/bin/spark-submit'' finished with non-zero exit value 1

• Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

• Get more help at https://help.gradle.org

BUILD FAILED in 4s Additional context I am using version 0.7.0. My spark version is 2.4.5, and my hadoop version is 2.7.7. From what I saw on other issues, the cause of this error seems to be incompatible spark version, however my spark version should be correct. I have tried using spark 2.4.7, but the results were the same. I did run ./gradlew compileTestScala installDist in helloworld folder and TransmogrifAI folder before submitting the example, and I am under helloworld folder when I submitted the command. I tried submitting it using $SPARK_HOME/bin/spark-submit --class com.salesforce.hw.titanic.OpTitanic build/libs/transmogrifai-helloworld-0.0.1-all.jar --run-type train --model-location /tmp/titanic-model --read-location Passenger=pwd/src/main/resources/TitanicDataset/TitanicPassengersTrainData.csv command, but the results were similar. My java version is 1.8.0_161, and I have set JAVA_HOME correctly. I did modifiy gradle/wrapper/gradle-wrapper.properties, so that I could use a local gradle of version 5.2.1. Also I have modified the repositories in build.gradle to two faster mirrors. I am not sure if this is related to this problem. I am running this example on a pseudo-cluster on a freshly configured CentOS7 virtual machine. Is this problem caused by the lack of some necessary libraries?

[tovbinm]

Which Scala version are you using?

[TechpriestW]

I think it was scala 2.11.8, but I can't remember very clearly. I don't have access to the device I was using anymore. I was pretty sure it was a version of scala 2.11 though.