search

sahandha / SparkClusterKubernetes

0 stars 0 forks source link

How do I get spark to use global IP addresses rather than internal IP for cluster set up? #7

Open sahandha opened 8 years ago

sahandha commented 8 years ago

When I set up a cluster, Spark uses internal ip (i.e 127.0.0.1) for slaves, but I need to use the global ip of the VM or at lease the internal IP of the VM (not the container).

Also, the master address becomes the container ID. How do I make that the global IP of the VM that holds the master container?

sahandha commented 8 years ago

I think this is done in $SPARK_HOME/conf/spark-env.sh.