Added /media/tmp for temporary usage that needs more space than /tmp

nchammas / flintrock

A command-line tool for launching Apache Spark clusters.

Apache License 2.0

637 stars 116 forks source link

Closed douglaz closed 7 years ago

douglaz commented 7 years ago

This PR makes creates a /media/tmp linking to /ephemeral*

A usage example is to set SPARK_WORKER_DIR to this.

I tested this PR by launching a cluster.

douglaz commented 7 years ago

Some testing questions:

Did you test this with instances that don't have instance store volumes (e.g. t2.medium)?

Yes, tested on m4.large

Did you test this with instances that do have instance store volumes (e.g. m3.xlarge)?

Yes, this was my first test

Did you test this with adding nodes to an existing cluster?

Yep

nchammas commented 7 years ago

Sorry about the delay in merging this in @douglaz and thank you for the contribution.