search

mahmoudparsian / data-algorithms-book

MapReduce, Spark, Java, and Scala for Data Algorithms Book
http://mapreduce4hackers.com
Other
1.07k stars 666 forks source link

Code uses yarn private classes so cannot compile #9

Closed axemaster closed 1 year ago

axemaster commented 8 years ago

Classes ClientArguments and Client in org.apache.spark.deploy.yarn are private and therefore not visible to user code. How are you getting this code to work outside of the yarn package?

mahmoudparsian commented 8 years ago

I am using the built single jar (by SBT) : spark-assembly-1.6.1-hadoop2.6.0.jar, which compiles and runs.

When I examine the source code for ClientArguments in NetBeans IDE, I see the following:

package org.apache.spark.deploy.yarn;

import org.apache.spark.SparkConf; import scala.collection.Seq; import scala.collection.immutable.List; import scala.collection.mutable.ArrayBuffer; import scala.reflect.ScalaSignature;

@ScalaSignature(bytes = "...") public class ClientArguments { ... public ClientArguments(String[] args, SparkConf sparkConf) { ... } }