Allow Spark/Hadoop download source to be an S3 URL

nchammas / flintrock

A command-line tool for launching Apache Spark clusters.

Apache License 2.0

636 stars 116 forks source link

Allow Spark/Hadoop download source to be an S3 URL #324

Closed nchammas closed 3 years ago

nchammas commented 3 years ago

Downloading from the Apache mirrors is slow. Most heavy users of Flintrock will likely want to host Spark and Hadoop download artifacts on S3 and have launched clusters download from there.

This change allows Flintrock to do that.

TODO:

[x] Update docstrings, config template, and maybe also main README.
[x] Add changelog.