below is the error log from the roject. The MoreExecutors.directExector() is available from guava v18 and above. However, the os-maven-plugin seems to have some form of dependency to guava 10/11 which does not have the directExecutor() method available in it.
Since the os-maven-plugin is defined as extension in the maven build, i dont see a way to define an exclusion of the older version of guava jar.
Driver stacktrace:
2017-11-04 22:37:47.234 INFO 74858 --- [launcher-proc-3] org.apache.spark.launcher.app.test-JOB : 17/11/04 22:37:47 INFO DAGScheduler: Job 2 failed: foreach at FixedLengthProcessor.java:75, took 5.287376 s
2017-11-04 22:37:47.235 INFO 74858 --- [launcher-proc-3] org.apache.spark.launcher.app.test-JOB : Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 2.0 failed 4 times, most recent failure: Lost task 0.3 in stage 2.0 (TID 5, 127.0.0.1, executor 2): java.lang.NoSuchMethodError: com.google.common.util.concurrent.MoreExecutors.directExecutor()Ljava/util/concurrent/Executor;
2017-11-04 22:37:47.235 INFO 74858 --- [launcher-proc-3] org.apache.spark.launcher.app.test-JOB : at io.grpc.internal.ClientCallImpl.(ClientCallImpl.java:94)
2017-11-04 22:37:47.235 INFO 74858 --- [launcher-proc-3] org.apache.spark.launcher.app.test-JOB : at io.grpc.internal.ManagedChannelImpl$RealChannel.newCall(ManagedChannelImpl.java:572)
2017-11-04 22:37:47.235 INFO 74858 --- [launcher-proc-3] org.apache.spark.launcher.app.test-JOB : at io.grpc.internal.CensusTracingModule$TracingClientInterceptor.interceptCall(CensusTracingModule.java:330)
2017-11-04 22:37:47.235 INFO 74858 --- [launcher-proc-3] org.apache.spark.launcher.app.test-JOB : at io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:104)
2017-11-04 22:37:47.235 INFO 74858 --- [launcher-proc-3] org.apache.spark.launcher.app.test-JOB : at io.grpc.internal.ManagedChannelImpl.newCall(ManagedChannelImpl.java:551)
2017-11-04 22:37:47.235 INFO 74858 --- [launcher-proc-3] org.apache.spark.launcher.app.test-JOB : at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:113)
below is the error log from the roject. The MoreExecutors.directExector() is available from guava v18 and above. However, the os-maven-plugin seems to have some form of dependency to guava 10/11 which does not have the directExecutor() method available in it.
Since the os-maven-plugin is defined as extension in the maven build, i dont see a way to define an exclusion of the older version of guava jar.
Driver stacktrace: 2017-11-04 22:37:47.234 INFO 74858 --- [launcher-proc-3] org.apache.spark.launcher.app.test-JOB : 17/11/04 22:37:47 INFO DAGScheduler: Job 2 failed: foreach at FixedLengthProcessor.java:75, took 5.287376 s 2017-11-04 22:37:47.235 INFO 74858 --- [launcher-proc-3] org.apache.spark.launcher.app.test-JOB : Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 2.0 failed 4 times, most recent failure: Lost task 0.3 in stage 2.0 (TID 5, 127.0.0.1, executor 2): java.lang.NoSuchMethodError: com.google.common.util.concurrent.MoreExecutors.directExecutor()Ljava/util/concurrent/Executor; 2017-11-04 22:37:47.235 INFO 74858 --- [launcher-proc-3] org.apache.spark.launcher.app.test-JOB : at io.grpc.internal.ClientCallImpl.(ClientCallImpl.java:94)
2017-11-04 22:37:47.235 INFO 74858 --- [launcher-proc-3] org.apache.spark.launcher.app.test-JOB : at io.grpc.internal.ManagedChannelImpl$RealChannel.newCall(ManagedChannelImpl.java:572)
2017-11-04 22:37:47.235 INFO 74858 --- [launcher-proc-3] org.apache.spark.launcher.app.test-JOB : at io.grpc.internal.CensusTracingModule$TracingClientInterceptor.interceptCall(CensusTracingModule.java:330)
2017-11-04 22:37:47.235 INFO 74858 --- [launcher-proc-3] org.apache.spark.launcher.app.test-JOB : at io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:104)
2017-11-04 22:37:47.235 INFO 74858 --- [launcher-proc-3] org.apache.spark.launcher.app.test-JOB : at io.grpc.internal.ManagedChannelImpl.newCall(ManagedChannelImpl.java:551)
2017-11-04 22:37:47.235 INFO 74858 --- [launcher-proc-3] org.apache.spark.launcher.app.test-JOB : at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:113)