search

yennanliu / AirflowJob

Airflow POC demo : 1) env set up 2) airflow DAG 3) Spark/ML pipeline | #DE
12 stars 5 forks source link

Run spark with Hadoop yarn / cluster / Mesos #4

Open yennanliu opened 5 years ago

yennanliu commented 5 years ago

ref https://jupyter-docker-stacks.readthedocs.io/en/latest/using/recipes.html#spark

yennanliu commented 5 years ago

mode 

1. spark stand alone
2. cluster
3. Mesos
4. Hadoop Yarn