search

JerryLead / SparkInternals

Notes talking about the design and implementation of Apache Spark
5.28k stars 1.84k forks source link

第一章配置多个 CoarseGrainedExecutorBackend 进程 #47

Open wojiaohgl opened 8 years ago

wojiaohgl commented 8 years ago

SPARK_WORKER_INSTANCES http://spark.apache.org/docs/latest/spark-standalone.html#cluster-launch-scripts 指定这个参数是不是就可以了？

JerryLead commented 8 years ago

SPARK_WORKER_INSTANCES 是配置一个node 上可以运行多少个Worker，一般一个Worker可以运行一个CoarseGrainedExecutorBackend进程，我的问题是如何配置一个Worker可以运行多个CoarseGrainedExecutorBackend

JerryLead commented 8 years ago

要想实现一个Worker运行多个CoarseGrainedExecutorBackend，需要将Worker的资源数设置为Executor资源数的N倍