wangqiaoshi
commented
7 years ago I found import org.apache.hadoop.yarn.conf.YarnConfiguration in HadoopJobUtils class.
i add hadoop-yarn-api-2.7.2.jar in jobtype.global.classpath,i will not throw YarnConfiguration Class not found error Exception
I don't need spark job running on yarn,but running on mesos. now can HadoopSparkJob support it?
java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/conf/YarnConfiguration at azkaban.jobtype.HadoopSparkJob.sparkJarsHelper(HadoopSparkJob.java:397) at azkaban.jobtype.HadoopSparkJob.testableGetMainArguments(HadoopSparkJob.java:342) at azkaban.jobtype.HadoopSparkJob.getMainArguments(HadoopSparkJob.java:309) at azkaban.jobExecutor.JavaProcessJob.createCommandLine(JavaProcessJob.java:65) at azkaban.jobExecutor.JavaProcessJob.getCommandList(JavaProcessJob.java:54) at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:93) at azkaban.jobtype.HadoopSparkJob.run(HadoopSparkJob.java:226) at azkaban.execapp.JobRunner.runJob(JobRunner.java:693) at azkaban.execapp.JobRunner.run(JobRunner.java:545) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.conf.YarnConfiguration at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 14 more