Why HDFS run without YARN ?

big-data-europe / docker-hadoop-spark-workbench

[EXPERIMENTAL] This repo includes deployment instructions for running HDFS/Spark inside docker containers. Also includes spark-notebook and HDFS FileBrowser.

687 stars 372 forks source link

Why HDFS run without YARN ? #56

Closed dolenam317 closed 6 years ago

dolenam317 commented 6 years ago

Could you please explain why HDFS is run without YARN because I think YARN is necessary for scheduling Spark jobs ?

Thanks,

earthquakesan commented 6 years ago

@dolenam317, Spark is running in standalone mode in the setup provided in this repo. YARN is not necessary in this case.